The full transcript is available and is also quoted below:
Dan Baker: So, you know, it’s kind of like building a car road – you build your own engine. And the answer is if you’re just wanting to get from point A to point B you probably wouldn’t, but if you’re a formula 1 race car you build your own stuff because it has to be competitive. And the honest truth is that Civ deserves a custom engine. We aren’t building a generic game, we’re building Civ, and so the engines on the market aren’t really suited to the type of game, so we really felt pretty strongly that we should build an engine that’s going to give the Civ player the best experience.
ET: Tim Kipp, Systems Lead on Civilization V, further explains more about why they created a new engine, and what the team gained from the endeavor.
Tim Kipp: I mean Dan is exactly right in that when we looked at what Civ V needed and we talked to the designer, and we talked to the lead artist, and we talked to everybody else, we realized that there was really nothing on the market that was going to be as scalable as what we wanted and provide the game player at the end of the day the experience that was really sufficient for what Civ V could be. So with their help and their guidance they let us go ahead and build it from scratch and the successful part about that for us is that we’ve been able to take the designer’s vision and the artist’s vision and we haven’t had to compromise yet in terms of what it is that they wanted. And that was really our – you know the goal wasn’t necessarily just could we get there as fast as we possibly could but could we actually create the vision that these guys wanted to have for the game and I think by and large we’ve been able to do that.
DB: And in my kind of geeky technical thing, there are really two things that we thought – that we really wanted to be able to do that we thought needed a custom engine. One is that we really wanted to make these random believable worlds. Unlike a lot of games or this carefully scripted experience, Civ is about randomness. It’s about randomness but a unique world every time you play so we really wanted to have these random worlds that looked as good as hand-made stuff. So there was no technology on the market – we had to invent a lot of stuff to be able to do that. And the other thing is that Civ is also a game about scale. I mean the game has a lot of stuff in it. I mean your world fills up with – Tim’s laughing – the amount of stuff that you can build it’s beyond what any game does. So we have to be able to – the needs of our game exceed most games by an order of magnitude. So that was the two major reasons why we realized we needed to build something custom.
John Kloetzli: I just have to say that this is the first project that I’ve worked on where we’ve completely built our own stuff from scratch and just as an engineer working on the system it’s been much easier than having to conform to someone else’s ideas of what you should be because you only build the parts that you need and you build them exactly the way that you need them. You don’t have to conform anything to someone else’s idea of what a game engine should be. Just from my perspective as a graphics engineer that was much less frustrating than a typical project.
Josh Barczak: Yeah it definitely gives you a lot of freedom during development that you just wouldn’t have if you started with an entire code base ready-made. You don’t always know what your exact requirements are going to be so the best way to build something that meets your exact requirements is to just do it from the ground up.
ET: That’s John Kloetzli, Graphics Programmer, and Josh Barczak, Senior Graphics Programmer, who were also integral in making the Civilization V engine come to life. By now, you’ve probably noticed a pattern emerging in what these guys are talking about – scalability being a big buzz word that keeps returning whenever they talk about the how and why of the new engine. Civilization V is, in fact, very, very scalable (amazingly so in my opinion. I can’t tell you how floored I was to hear about the gamut of machines that the game will run on.) Making the game so resilient to a wide swath of different computers – from commuter laptop to beefy gaming machine – must have been a lofty goal.
TK: I think there was a lot of careful planning that went into the design of the engine very very early on. I think there was also a lot of sort of very honestly looking at how games are produced, what needs to go into it. So effectively we’ve probably pulled away a lot of the extra glue and a lot of the extra (inaudible) that wind up weighing down a lot of engines and what we’ve got is something that’s very clean. Very small set of code that does exactly what we want it to with very little to no side effects. And that’s something that Dan designed the graphics layer from the ground up to be completely like that. And then we’ve designed the rest of the engine to be very very similar in that everything is there for a reason. There’s nothing there that doesn’t belong there and there’s no code that gets executed for no reason whatsoever. And that’s allowed us, at least in terms of efficiency, to scale very well on a single core. The second side of that is that Dan and I looked very hard at threading scalability in terms of how games typically wind up implementing that. Most of the games and the game engines you’ll find out there are very functionally threaded which means they’ll run physics on a thread, they’ll run graphics on a thread. It’s very very coarse. Very very coarse and beyond say the Xbox that doesn’t scale very well. Once you get to quad core and beyond you just wind up running out of functional elements and you wind up requiring to subdivide. However if you design for a functional paradigm what happens is that trying to break that up into smaller pieces now introduces additional overhead. So we design everything from the ground up to be job-based, task-based, very well encapsulated so we’ve scaled up to, what is it, 12…
DB: We’ve tested on 12 threads but, I mean, we don’t have anything more than 12 to test on but we’re pretty confident it’ll work with a lot more. Yeah it’s amazing how it spreads across the whole system.
JB: And I think this also goes back to developing something that does exactly what we need it to do and what we want it to do. The more general – the engines that are out there on the market - the commercial engines – they’re designed to be as generic as possible to cater to as many different possible customer configurations as they can and one of the drawbacks of that is the more general you try to be you inherently lose some performance as a result. You’re going to be less streamlined. You don’t know exactly what you’re going to do up front so you kinda have to prepare for the worst case.
DB: And I always liken to – my brother does autocross and he has his own little (inaudible) that he races and when you race on the track, you don’t buy like a car off the street like a Civic or something because, you know, it’s not designed for that. So his car doesn’t have a lot of horsepower but, you know, the backseats are gone, there’s no air conditioner, the power windows have been taking out, the tires are exactly special custom, and the thing performs orders of magnitude better. Now it’s not a general case, but we’re just trying to do something specific and you can get much much better performance and quality when you know this is exactly the type of thing that you’re gonna be doing. And I just liken it to, you know, making a sports car or making a bust or something.
JB: I think the other factor too is there really is a culture on this team of making the code as lean and as streamlined as it can possibly be. Which, paying attention to the small details as you’re going along, little things add up. There are a lot of little things built into the game.
JK: Yeah I think on this project one of the most valuable lessons is that the senior engineers set the tone for the project and if they’re really concerned about the performance and really concerned about code optimization, constantly coming to the more junior guys and saying, “you know this is good but it could be a little better if we did it this way, a little better if we did it this way” - really sets the tone for the project and I think that’s one of the main reasons why we’re able to get the performance levels that we do get is because a lot of the senior engineers have really set that tone and it’s fallen through to the rest of the engineering team.
ET: Clearly, building the back end of the game took many more people than just the four in front of me – as with everything in Civilization V, the team worked as a cohesive whole to make the dream a reality. And in doing so, the team made technical innovations that they should be very proud of. And since so many gamers (myself included) so often never get a chance to look under the proverbial hood of a game and marvel at what makes the thing go – I wanted them to explain it to me.
DB: One of the nice things about working here is that our secret sauce is making the game great. Gameplay great. So the cool thing about that is as a graphics guy I can actually talk about what we do because we don’t consider that our competitive edge. And gosh there’s so many things technically. Like I had to talk on Tuesday on all the stuff and you forget all the things. So I’ll talk about a few of them and these guys can elaborate on some stuff that we did. There’s like two different main modes to the game. There’s the main game that you see and you’re playing and there’s this other whole thing where you go in and you actually talk to the leaders. And that’s – they’re completely different and rather than try and make the same engine part that does both of them they’re actually quite a bit different and there’s a lot of technology that we threw at it. The first set of technology we did is a lot of stuff – and I’ll let John elaborate on some of the stuff that he worked on – was to make these custom worlds and blends. So the gameplay makes a custom world every time you start so one of the goals I had was to build a map that really looked like someone hand-painted it. And there’s a lot of subtle technology dealing with that – a lot of procedural things and John if you wanna elaborate on…you did a lot of work on that.
JK: Sure, yeah yeah Dan and I worked on that from the beginning. The terrain in the game is one of the oldest systems – in fact it might be the oldest system in the code base, the oldest Civ V system. So really what Dan and I did – we kind of sat down and said “how can we design this – add in limitations to the way the artists build the art so that we can build things procedurally?” So essentially we have the artist go through and build a large database of pieces which we can then map to whichever map script the designers hand us to build this randomized map on the fly. Because we have a large database of pieces we can pick out special configurations. Like, “oh look here’s three mountain pieces that end up, you know, lined in this direction.” We can pick a special hand-made piece from the artist and fit it in there but it’s like playing a big game of Tetris – you don’t know exactly what’s going to be next to you on all the different sides. So we spent a lot of time working on technology to get all these pieces to blend together correctly and then light them and do shadowing calculations and all those things.
DB: One of the stories of the project has been this kind of symbiance between procedure and art direction and we really embraced that. For instance, one thing that happened - it was like a year ago I think about – where John comes in my office and the artist is just having trouble making these coastlines because they have to match up all this stuff exactly. I think we can do it with a spline. I thought about it for a minute and I’m like, “all right, try it. I think you’re right I think we can do it with a spline.” So the coastlines and the rivers actually they’re called program art. The program actually makes those because it turned out to be so labor-intensive for the artists to do it. And the artists were ecstatic because there’s this huge amount of work that they were looking at to making perfect coastlines where we said, “you know what, we think we can make the computer do that tedious job for you.” And there’s many other cases I won’t get into that we procedurally handle stuff. One case, for instance, is the ice that we’re putting in the game is also procedurally done with like artist influences. And, again, because it’s a random world you can’t hand-build this stuff. You have to have unique stuff so we’ve really embraced that and that’s a very unique Civ.
TK: It’s also important to emphasize the fact that not only do you do that procedurally for the artist that the artist still has complete control over the outcome which is I think one of those cases where you didn’t just stop, at the “we can generate this procedurally” but you then went the next step further which is really important which is, “how do I give the best control possible to the artist to get the result they want?” And, I mean, that’s true throughout the entire terrain system. I mean everything that’s there shows up the way the artist wants it to. So it’s not just about the fact that we can generate these things procedurally but it’s that we can capture what the artist wants to do and then reflect that accurately back.
JK: I think that’s one of the big differences between the way the research community looks at procedural generation of graphics content and the way that you have to do it in games. A lot of times – and it’s gotten less so over the years – but the research community tends to think of things as completely procedural. I press a button and the terrain comes out the other side.
DB: George Jetson.
JK: Yeah but that’s not really useful for us. We have to have controllable procedural. So it’s something that the code is filling in all the details but it’s controlled at a high level. Very tightly controlled at a high level by the art. We want exactly grassland here and exactly coast here, and that sort of thing.
ET: The balance of art and realism, planned terrain and random, fresh worlds with each new game… Building an engine that expertly executes all of these elements is incredible in its own right. But Firaxis didn’t stop just with that aspect of the engine. In Civilization V, there is an entirely different view beyond the world map – when you approach leaders to wage war, negotiate peace, or barter for important resources – you enter a completely different view, and its visual fidelity is astounding. The way the leader engine works and was built is completely separate from the “main” view of the game.
JB: It’s another advantage of having a custom engine. The leader viewer is – Dan likes to jokingly call it the first person shooter that we’re embedding into Civ V. It’s really a very different system. It’s got rendering needs that are fundamentally different than the rest of the game.
ET: You don’t shoot the leaders.
JB: No you do not shoot the leaders.
DB: This is our April Fool’s Joke.
TK: There could be a mod. There could always be a mod.
JB: What I like the most about it – what I’ve enjoyed the most about working on it – is that it’s essentially a sandbox. I have the entire machine available to me during that time, provided I don’t use up too much memory, and I really get to pound on the GPU and to throw as much extra shading and as much extra visual quality into it as I can. It’s a great platform for us to really show what we can do graphically.
DB: And it’s funny because we got a few comments here and there – people thought they were movies. And like “no those are actually – they’re not. They’re rendered in real time.”
TK: Well when we first started, you know, we had concept artists that painted these entire scenes and everyone looked at those scenes and said wow those scenes are awesome. And we looked at that and said, “well is there any possible way that we can make that scene look that good in game?” And we had some visitors in I guess it was probably a month ago, and some of the leaders on the wall – we have a giant wall of leaders – half of them were concept art or pre-rendered stuff and the other half were in-game screenshots. And everyone that looked at them had – it took them at least, what, 10 seconds maybe to figure out which ones were which?
DB: Oh yeah it was – we were like, “pick which ones are in-game.” And they’re sitting there for a while, looking at them, looking at them. And it wasn’t obvious what was actually being rendered and what was the concepts. They didn’t look like what we’re used to seeing in games so we were pretty happy about that.
JB: And there’s another key area of technical innovation in all of the compression stuff that we’re doing for the leaders.
DB: Yeah we have a lot of advanced technology in the engine too like a couple publications or one publication probably some pending publications – like the water rendering for instance is pretty high tech. A lot of it is about subtlety. You know people sit there and you don’t realize it but we’re not about making things flashy. A lot of games they want flashy and I don’t know what you would call it – bling, bling is probably it. I got the bling, and we’re all about not having the bling. So there’s a lot of technology in making it not look blingy and just looking good and you’re not gonna realize yeah we spent a lot of time making sure that the water didn’t alias, it didn’t shimmer, it looked correct when you zoomed in and out. And there’s a subtle distance fog that kind of maps in at the edge of the world that’s supposed to be very controllable. And we put a lot of time into that and yeah it’s not bling, but all of that kind of adds together to give a unified experience.
ET: One of the biggest questions every developer has to grapple with when creating a game is “do I license, or build from scratch?” As with everything in life, there are always limited resources, and limited time, when building a game – so you can’t always do everything, everywhere, amazingly. Firaxis’ decision to make a scalable engine that procedurally generates gorgeous graphical images (in two very distinct ways) is a monumental achievement for games. They didn’t have to go this route. There are dozens of ways that they could have made this task easier and still created a product that would have created a jaw-dropping experience for everyone, but instead, they went far, far above and beyond and made something veritably mind blowing. So far, after showing the game at GDC, E3, and to select press over the past few months, the response to this decision has paid off – people love the game. How did the team think the industry, as a whole, would respond to what they had built – and think about their choice to create rather than license and augment?
DB: Good question, why build your own thing. I hope that from a technology standpoint, and I think people already do and I’ve talked to – I know a lot of the graphics leads and a lot of perks of the industry, a lot of friends of mine and we trade war stories. We go to GDC and you’re on the table and you’re telling a war story, “look what my artist did to me” and you know, scar here. “Look what my artist did to me.” And it’s interesting hearing what their problems are and one of the things that kind of struck me is how we had built technology in certain areas that just far exceeded what people were telling me was possible. They said, “well you can’t do that” and I’m like, “but we are.” And they’re like “how are you doing that?” And I’m like “I don’t know how you’re not!” We don’t understand like where certain situations where our performance is just beyond what even the hardware guys are telling us “but you can’t do that” and we’re like, “but we are.” And so in some situations I hope that the response is, “oh wow look what you can actually do.” A lot of times people say something’s impossible until they see it done and then it’s, “oh wait, yeah that could be done.”
TK: Yeah I’d like to think there will be a lot of positive response from it. And generally the response I think traditionally depends on who you’re talking to. In a lot of cases, engineers that have been in the industry a long time, they know if you’re building something custom that you need to build something custom with it. When you’re building a project that has a lot of specific requirements in it, trying to pull something general and make it perform to the characteristics that you need, typically takes longer than doing it the right way the first time. So I think if you ask a lot of experienced engineers they will say that this is probably very very obvious to them and why wouldn’t you do it this way. Various marketing people or somebody else might have a different response especially if they work for a company that would like you to buy their technology. And if you work with a lot of people that have never actually built their own stuff before, a lot of times they only understand the works that they’ve worked in before. So it’s, you know, in some cases when you’re breaking new ground it’s hard to get people’s perspective to change a little bit in that regard. But I think we’ve gotten a tremendously positive response so far.
JB: Yeah that was actually one of the questions that I had when I first came here actually is “why build your own engine?” You know, historically Firaxis has used Gamebryo and has shipped many successful titles that way. Why go branch out and build a custom code base? And I think the answer is that there’s at least as much engineering effort in either path. Your game is still your game. It’s still unique and you’re still going to need to spend time adapting what you have to what you need to do. And there’s just so many things that become easier if you can build the entire thing the right way the first time.
ET: I’m Elizabeth Tobey, and we’ll be back next episode to discuss Civilization V’s user interface.
Origineel Artikel: http://forums.civfanatics.com/showthread.php?t=372022&goto=newpost