#028: More Than Testing: The Truth About Game QA

00:00:02: Hello and welcome to the answer podcast.

00:00:02: My name is Paweł Buża, And I am joined by Paweń Mielniczyłk, The

00:00:05: art director

00:00:06: for project Haydar.

00:00:07: Pawew who do we have today?

00:00:09: Hi!

00:00:09: Today our topic will be about quality assurance.

00:00:12: Our guest would be Monika Rockitar narrative QA lead and Piotr Tyszko a sweet QA Director.

00:00:19: It's gonna be an incredible episode... ...and i can't wait to meet our guests so let's meet him.

00:00:27: So Monika Piotrz.

00:00:28: Welcome to the podcast.

00:00:29: officially QA one of my favorite topics.

00:00:34: I'm surprised we haven't done this earlier, to be honest because it's a foundation.

00:00:40: i would say

00:00:41: you

00:00:42: know?

00:00:43: We'll just blame it on kalemba!

00:00:45: It is the foundation.

00:00:46: but before we get into all that could you guys introduce yourselves for people listening so they can hear your voice and associate with the person Sure.

00:00:57: So I'll go first, I'm Monika Rokita and I am a narrative QA lead working in Reds since two thousand thirteen with the short maternity break.

00:01:07: leave And always think you way started in qa.

00:01:10: start it in narration.

00:01:11: still in QA.

00:01:12: still narration after all those years because i like It obviously and that's it?

00:01:17: I guess

00:01:18: amazing

00:01:19: alright.

00:01:19: I'm Piotr Teshko associate QA director.

00:01:22: Its now over seven years in CD project In general, however it was over five years in geology and almost two in red today.

00:01:32: Perfect!

00:01:32: I'm happy that he said project not project because...

00:01:36: That's how

00:01:36: you say it properly perfect.

00:01:39: So guys let's start from the basics.

00:01:41: uh i think when people listening or watching this they think QA For sure, these are the people who just sit down and play game all day.

00:01:49: I could do that right?

00:01:50: Anyone can't because it's easy but i feel like its a big misconception.

00:01:55: so what is your take as to actually working QA?

00:01:59: how does you're day-to-day work?

00:02:01: And this is sitting out playing games over an hour.

00:02:05: So why think I can talk about this one This kind of my day today?

00:02:09: Yes thats true.

00:02:11: this most common misconceptions in QA Thats true.

00:02:14: And yes, we play the game.

00:02:16: But not in a context that you will think of.

00:02:18: it's like... We do this for fun as well because we have to make sure that the games is fun but also analyze them and there are many aspects of playing these games.

00:02:32: The analysis based on many things which need to be checked.

00:02:37: There're subjective stuff.

00:02:40: so we played the game but its'nt only thing And we are not the only ones doing that.

00:02:46: Basically, everyone in their company should play it a game at least once.

00:02:50: so yeah...

00:02:50: Yeah I think very often looks like you're playing just as said right but A lot of things happen behind-the-scenes and sometimes use this metaphor solving the puzzles under the microscope.

00:03:08: It can look like we stare on same door for forty minutes But actually it values, right?

00:03:14: This is QA also.

00:03:17: Yeah so it's more than just looking at a door but sometimes

00:03:21: there are like little pieces you don't see yourself and they're important

00:03:25: like from different angle and with a different context in mind, but this is the same door all the time.

00:03:32: I think the other misconception could be that QA's did last step right?

00:03:36: So we make the game then send it to you guys... You test it!

00:03:40: And its done

00:03:41: so.. Totally the opposite.

00:03:44: The approach were now having kind of looking forward to have in future was early into development

00:03:50: Right?!

00:03:50: This is terminology for software QA shift left where basically we try to be as early as possible.

00:03:59: So, you know it's always easier to move a chair or this door I mentioned on paper than later once the house is built around the chair.

00:04:08: You know what i mean right?

00:04:09: so It's better to be earlier to prevent from like a costy and big bugs and problems in the future Right?

00:04:16: I remember from Funtal Liberty because previously I was mostly engaged in the characters and since Phantom of Liberty, also working environments.

00:04:22: And it did not realize for example how many elements connect to environment so that scenes placement dialogues you know unique points all this stuff... Many

00:04:32: systems are overlapping, many futures are overlapping and we're people

00:04:36: having good QA there with a team from very beginning with the content team level design with environmental artists.

00:04:43: We've seen designer just taking on top of all these.

00:04:48: moving elements was incredibly important.

00:04:50: We are even earlier in the process, not only when the game is already implemented and the puzzles being done.

00:04:57: we're even at a design phase because we also have our other side... We are the guardians for quality.

00:05:04: QA means Quality Assurance so you need to guard your quality and support it many different ways Not just when the games are done but when there is an idea of the game and an idea for a feature or system, everything needs to be working together.

00:05:19: We are there... ...to be voice.

00:05:22: ideas risks many other elements that we see already at this point.

00:05:28: so from beginning basically

00:05:33: it's like sometimes people associate QA with just finding bugs right?

00:05:37: Yeah I was gonna

00:05:39: say Only this is really important of course, but beyond box bugs are visible.

00:05:45: But beyond box.

00:05:47: there was like.

00:05:48: I think that the main goal of QA Is to protect and prevent from like a bad player experience.

00:05:53: so we protect player experience And we try to spot identify and reduce possible risks.

00:06:00: Sometimes we need to jump to the fire.

00:06:02: sometimes it's an as better scenario To go where the fire will be in the future always cheaper You know,

00:06:11: where the fire

00:06:11: is.

00:06:12: Then fighting in the fire right?

00:06:13: So that's what we do and what Monica said about QA.

00:06:19: like you making sure there was a quality of game.

00:06:24: That's our role.

00:06:25: I try to explain it too The people on QA also outside QA company that QA is one accountable for quality.

00:06:33: But QA cannot be only responsible for quality.

00:06:35: It's common effort, teamwork.

00:06:40: There are producers, designers artists engineers all the development teams and folks who collectively work on their quality.

00:06:48: But we drive that like quality aspect right?

00:06:52: We make sure they see the risk areas They identify the problems with try to address them as early as possible.

00:06:59: That's our job.

00:07:01: And one more thing.

00:07:01: sorry I'm talking too much but One more think about QA To show to the people like there are two coins.

00:07:11: Sorry, two sides of this coin right?

00:07:13: There is objective QA.

00:07:17: I would say it's more about functional testing and making sure that the game-the product works.

00:07:23: But there was also subjective QA which is more about The experience.

00:07:31: how does it feel to play a game doesn't match the general Game direction and vision that we decided.

00:07:37: We want to have in this particular game, right?

00:07:40: And it's important to blend.

00:07:44: blend both of them is like once you have the car will make sure that the car has brakes ride but then we need to check if he feels It doesn't feel like a shopping car going downhill.

00:08:00: You know, it has to be smooth drive right?

00:08:04: There

00:08:04: are some cars that way.

00:08:05: I know beaters

00:08:07: but you also have like to have like the house to have pretty color.

00:08:12: So make sure that the people looking at it also.

00:08:14: Yeah, you know there are many emotions connected to that.

00:08:17: There's things like checking pacing of a game checking motivations of their characters Many things that they're like zero.

00:08:25: one thing Not technical wise not only about working but being fun again, the fun thing.

00:08:34: But yeah this is important also

00:08:36: I think what helps because how i think about QA is those two things that support it like.

00:08:40: first of all there are most experienced players overall, gamers.

00:08:44: They are working libraries of all the games and you

00:08:47: know... We have to play the game.

00:08:48: we have one thing for the staff, what is going on in the market?

00:08:51: What people want with a player wants.

00:08:53: so

00:08:54: I would say also that best players right because like players gameplay wise yeah.

00:08:58: So there's this experience.

00:09:01: they can share these experiences from different games.

00:09:04: exactly The developers who not always as experienced gamers but do it.

00:09:09: But second things Those are people in the company who know them most about game, from all layers.

00:09:15: From engine design to final experience through narrative because a lot of people in departments they're very specialised and locked inside their own... And like the word siloces but they focus on part of work.

00:09:30: QA is something that's the glue.

00:09:32: Weird

00:09:33: collective.

00:09:34: I wanted to say that I fully align with what you said.

00:09:38: Yeah, we are the first audience and very often.

00:09:41: The first critics of the game right?

00:09:43: And to be this critics in like a You know meaningful and healthy way We need to be experience what you said with the games rights for example if we want to have Some touch over I don't know souls like in the game.

00:09:59: We need To know those souls likes.

00:10:00: we need to play them.

00:10:01: someone has to be really experienced in that because We are not trustworthy partner if we don't, right?

00:10:07: If you want to have an opinion this has to be based on our experience.

00:10:11: So yeah QA has to play a lot of games.

00:10:13: What about specializing?

00:10:14: because we kind of touched upon it.

00:10:16: so... ...if he could tell us more or less like.

00:10:18: if you look at QA as whole You said your mainly narrative but there's different parts of QA stuff that focuses on specific part of the game.

00:10:27: what is division?

00:10:28: pretty much

00:10:30: It's complex.

00:10:30: maybe you can start with yours first.

00:10:32: It is complex.

00:10:35: paint a picture.

00:10:36: On the project that I work on, because every project can be different right?

00:10:40: QA is adjusting to the project and to the needs into the areas that are created in the projects.

00:10:45: so for our projects in red we have those five four or five main areas its narration it's gameplay it's tech and art an open world as a fifth one And QA are justing like more focused on what those areas are creating, implementing.

00:11:06: So we have this deeper knowledge about stuff because it's important that we work as an analyst and analysts need to know the tools they needed to know systems in process.

00:11:19: We need everything that designers sometimes even more And then you'll see a broader picture of holistic view Because the system is overlapping with each other And, you know we are just specializing in something.

00:11:34: We're better than some thing but it doesn't mean that were only in this thing.

00:11:39: if I'm a narration and they work mainly with quests and cinematics and story the things there building generation most of them mean that i don't want to quit level design or localization because those are also an ingredients.

00:11:54: There's a metaphor that I have to use, this is the point where they'll use my super metaphor Because i think QA is like tasting critics.

00:12:03: Like restaurant critic we're tasting the dish somebody else prepared and checking if their ingredients work with each other If there is enough salt from level design Enough sweet From story side And stuff like that being specialized help us to do it better, basically.

00:12:22: But It also doesn't mean that we only like you know have the view um... Like We have to have a broader view.

00:12:29: Yeah can

00:12:29: I be tunnel vision on one side?

00:12:31: yeah..we

00:12:32: have To Be careful with this because we don't want all Also to be silo in our Disciplines or areas And especially when now we are developing multiple projects i really encourage ourselves to be more flexible and adaptive in different scenarios.

00:12:49: So yeah, this differentiation that Monika said is like in place.

00:12:53: but we also need to remember to be adaptive right?

00:12:57: One more thing I want say which was really important... This division for example form our team at Polaris when we share a four game.

00:13:07: But uh.. We can't forget about the rest of QA Which are also disciplines such as localization QA Audio-Qa QA engineering, which is part of TechQA but also recently formed a really important team publishing QA.

00:13:23: We're supporting our external projects where we have different type of work because we are not like a deaf QA team embedded to the content teams in-house project But we kind of verify the work of the developers External developers who worked on the projects for us right?

00:13:40: Because we became publisher as a company As you know Totally.

00:13:44: So there is like a big spectrum, making sure

00:13:47: that the external projects are matching our own quality and

00:13:49: exactly.

00:13:50: so yeah once we decided okay let's publish The Games That Are Developed By Someone Else But Let'S Make Sure This Meets The Same Level Of Player Experience.

00:13:59: We Used Our Players Right?

00:14:00: So We Don't Screw It right.

00:14:02: this Super Important And Actually So Far Is So Good but uh...but Will See In The Future Right?

00:14:10: And does the QA adapt to phases of projects?

00:14:14: How do they change over production?

00:14:17: Good question.

00:14:18: It doesn't, it doesn't!

00:14:21: This is a complicated question because those... Again I'm speaking only from my perspective on project and bigger one our inside project Stable.

00:14:33: there are their and we need to work with those areas, but the needs change like that.

00:14:38: The needs from this area is changed And obviously they're different phases of the project in different needs During those phases as you know at the beginning or design.

00:14:47: so There's a bit lower need of QA.

00:14:50: But if you go further into game other things Like compliance that stepped it in So more things are

00:14:58: important thing.

00:15:00: So Compliance QA is a QA that verifies and checks if the game, and the feature in-the-game are in line with certification requirements for each platform.

00:15:11: For example you need to publish... You're about to launch the game on PlayStation five Xbox or Nintendo right?

00:15:19: And each of this platform has different requirements.

00:15:22: we need to meet all of them.

00:15:24: Yes, I'm really proud of our compliance QA team.

00:15:26: It's part of our publishing QA.

00:15:28: We have one men army now but supported by external partners who support us in testing.

00:15:35: So far every certification we had since like twenty-twenty four was successful right so in first attempt which is a big success In my opinion and this is important to have someone in the studio in qa where we need and how to get prepared for the certification.

00:15:52: It's not like last

00:15:52: thing that

00:15:57: you do during development, but in early development or pre-production.

00:16:02: it needs to link with engineers about how game will be built to meet these requirements at the very late stage of a project, right?

00:16:11: We can act earlier than just waiting for this moment.

00:16:16: But you need start thinking early is it's?

00:16:18: I think that This Is The Essence Of QA.

00:16:20: You Need To Identify Some Important Issues Early Because Later On In The Late Stage Of The Project Like I Don't Know Advanced Production It Could Be Too Late Right Or Too Costy.

00:16:32: i Think That This Could be One Of Mistakes That The New Studios Are Doing.

00:16:35: For Example There thinking about introducing QA is too late in the process, right?

00:16:39: Thinking this like it's a last step now.

00:16:41: The game has done basically sit and tested.

00:16:45: yeah I think

00:16:45: you used to be like that for years in industry.

00:16:50: It changed long time ago probably in US changing also Europe.

00:16:54: i'm glad This Is Happening because we see benefits of.

00:16:58: Yeah, it needs to be an iterative process like with anything that is in within game dev.

00:17:02: Like you need to happen at the early stages of the game and then just going through different iterations testing I'm playing kind of finding out what's next thing?

00:17:10: That should be...

00:17:12: Just one more thing.

00:17:13: It's like with QA sometimes you don't see...it's like we're producers.

00:17:18: You don't say sometimes The value of production or QA unless you Don't have it right?

00:17:25: Unless he misses They're

00:17:27: like, yeah we're screwed pretty much.

00:17:30: You also mentioned that you use external QA partners.

00:17:34: so how often do we actually reach out to external partners with help and How does the collaboration work?

00:17:41: Quite often very often I would say All

00:17:45: of our

00:17:46: projects maybe not all of them We have at the moment were multi-project studios.

00:17:50: So we worked on a small project sometimes And then it's okay just for us to have a couple of analysts inside.

00:17:57: But with majority of our projects, we use external partners who are responsible for functional tests mainly sometimes compliance test.

00:18:05: These are testers.

00:18:06: this is the difference.

00:18:07: they're testers were analysts inside our studio and you know within like complexity of the project.

00:18:17: at later stages We need to scale up our testing And then we have more testers right.

00:18:23: so We use testers, we're really glad that we have them.

00:18:28: So thank you our partners and we have you on board.

00:18:32: Because

00:18:32: both roles are important.

00:18:33: like there's different roles.

00:18:35: Analyze more, testers play a lot of functional tests but both roles is very important And obviously working with each other

00:18:42: I think that very often external testers The main source of knowledge about the state, and its functionality.

00:18:53: Sometimes they're the supplement.

00:18:55: sometimes we really trust the value that you take in your data.

00:19:00: We have thanks to their work And try use it To make best decisions About potential risks or how we develop our game In general.

00:19:13: So let's now maybe dive deeper into what the actual work looks like.

00:19:17: Monica, can you tell us more about this narrative QA?

00:19:19: That is interesting for me because I think people might have different imaginations.

00:19:23: What does actually mean to be a narrative Q A?

00:19:27: What it means!

00:19:28: This is a philosophical question.

00:19:32: I already mentioned that the narration works with specific themes.

00:19:36: The core of our job is working With teams who create its narrations so basically quests story, writers and cinematics.

00:19:45: But we work with everybody like the teams are... Piotrk mentioned being embedded.

00:19:50: We didn't explain that but this is important actually in this situation because The specialization also means for us in red That we're embedded.

00:19:57: that means we are working close With those groups with those teams with those developers And when I mean closed i mean very close Like on meetings On plannings All other things there like being decided.

00:20:12: we are there.

00:20:13: Super important

00:20:14: Maybe when you're not the decisive people, We were more of supporters group but still they and we can still have a voice about it.

00:20:23: So in narration... We obviously work on quests.

00:20:27: so play game.

00:20:30: Our most important aspect is playability of games.

00:20:33: You start then finish it And move through the quest Pick your own choices and finish it.

00:20:39: And there is no blocker on the way.

00:20:42: So this is the most important aspects from that for narration, and There are many many things.

00:20:48: We're going into display ability things and again The soup thing so many ingredients coming in to the soup and we are testing It and a feeling if the story works well If the motivation of the characters works Well, here's a continuity.

00:21:03: if you were doing like multiple titles one by one.

00:21:08: And also between quests, right?

00:21:09: Because each quest has different quest designers and sometimes maybe they can get lost in their own work and not forgetting about interconnecting everything together.

00:21:18: We need to take this holistic view of the game... ...and know if we are comparing a lot of stuff.

00:21:25: The pacing thing that I mentioned is checked along with full-game.

00:21:30: Each scene can have its own pacing but also each quest can have this, but also the whole game has their own pacing.

00:21:37: So we have this whole level thing of checking displayability stuff from beginning to smaller parts till end and what's important is that in our work Again, soup thing.

00:21:57: Everybody's implementing everything into this one big jar with the soup.

00:22:02: so we are The last people looking at it.

00:22:04: So not only the narration thing but also if other features gameplay features worked well with this narration.

00:22:11: So you were like a cook tasting the final product, right?

00:22:15: Yeah exactly.

00:22:16: Each ingredient could taste delicious but together not always!

00:22:19: Right?

00:22:20: Exactly

00:22:21: yeah I think that's really important to think about your side of QA.

00:22:24: Your area is narrative.

00:22:26: our games are narrative-driven game so This is super important for us.

00:22:30: To have this aspect and focus on this Not only this It

00:22:35: needs to be perfect.

00:22:36: Yes, this is something that makes us unique.

00:22:39: basically

00:22:39: We like to do many many paths and many choices so UA needs to fill the consequences in game.

00:22:47: That works.

00:22:47: Starkey even before you said on paper I know Quest designs.

00:22:52: they usually start with massive documents A, B option.

00:22:57: If you do that this happens.

00:22:58: we have read there's a lot of I guess

00:23:01: branches thing

00:23:02: going through all of that making sure these logical connections are working etc.

00:23:07: This is our job exactly.

00:23:09: and There is a think called like path testing the player path testing And this also The most important thing next to the playability for narration.

00:23:18: So We make sure That they're all choices at the player.

00:23:20: take among the game Like All the variables that You can Take are finishing well, like they're not blocked or not stupid basically.

00:23:29: So you do something and there is a consequence that doesn't fit your action.

00:23:33: so we play the game.

00:23:35: but we played in context with hundred steps to check if every variable... If take car right now for just fast travel some place it sounds very monotonous, it kind of is because you have so many steps that you have to follow and there are so many possibilities for those paths.

00:23:55: we cannot even cover all of them.

00:23:57: There's millions of these possibilities the player can take but try to cover as much as possible.

00:24:04: And yeah... We do this a long time on our project like if game is ready we're checking whether all the paths or possibility variations are correct Fun again, and working basically.

00:24:21: That must be super difficult because we work on massive RPGs And the games are pretty huge like if you look at Witcher three a hundred hours or more easily.

00:24:30: so Are there any shortcuts?

00:24:33: Or Any things that you can do in order to speed up some thing?

00:24:37: So kind of?

00:24:37: I don't know.

00:24:38: move faster traverse faster Like you have a big open world too.

00:24:42: Kind of traverse.

00:24:43: Is it harder?

00:24:43: any tips and tricks that you Can pretty much do In order make the process a little bit faster.

00:24:49: Yes, but they're based more on what the engine can do for you.

00:24:53: so we in our studio prepare this stuff inside.

00:24:57: So if there is any need to skip something We have engineers or people creating these things For us.

00:25:04: There are many debugs.

00:25:06: actually This work didn't come up today.

00:25:09: Debug's We are going deeper into the stuff.

00:25:13: we're not only just you know analyzing what it's on the shallow ground is looking at the back.

00:25:18: This isn't working and that's it, we were going deeper in debugging.

00:25:21: why its not working?

00:25:23: so there was a lot of things to cover.

00:25:27: I was actually um... The person who was preparing for cyberpunk thing and for Phantom of Liberty.

00:25:37: And actually, I was working on the paths for Witcher three.

00:25:39: so... I'm a good person to ask about it!

00:25:42: There were many variables that we needed to check.

00:25:46: Actually my favorite thing they like to mention from Witcher Three is when you wanted all possibilities that Quest Designers prepared.

00:25:56: We didn't have much time obviously because there could be thousands variations.

00:26:01: The QA as narrative QA.

00:26:05: We wanted to check it in pairs, and one person played during the day.

00:26:10: And the other doing the nights.

00:26:12: so we had that night shifts.

00:26:14: It isn't normal if you don't have a night shift normally?

00:26:24: For example, the Baron thing in Witcher three.

00:26:26: All of the possibilities that you can play this quest thanks to Pavel Sasko.

00:26:32: Pavel?

00:26:34: As we

00:26:34: mentioned!

00:26:36: We needed to check all those things and... ...those massive games-we like to do a massive game.

00:26:44: so give people their choice.

00:26:46: To make them roleplay on our games.

00:26:50: We need to make sure that this is working.

00:26:52: Yeah, giving them the freedom.

00:26:53: I think it's a big thing

00:26:54: but also fun for us to play game and check what are different things.

00:26:58: just

00:26:58: one more thing to add here of course when our games as you mentioned very massive big game so we're not like able play them all the time, you know like to run the game from A to Z. So we do.

00:27:11: also the speed runs right?

00:27:13: Oh yeah but because there are different type of paths that you can take We're also doing that.

00:27:17: There is a completionist path The biggest one.

00:27:20: That should when You do everything or the you Know skill league question mark

00:27:26: Gotta get platinum.

00:27:27: I mean come on.

00:27:28: Yeah i'm Also a platinum gamer so

00:27:31: I

00:27:31: did the platinum things in the PlayStation made those completionist paths to make sure that you do everything, but also there are speedrun paths.

00:27:41: To make sure they can skip every thing.

00:27:43: There's the failed path on the way because it fails the quest.

00:27:47: So again many different possibilities we need to think about and many tools for doing this Because a lot of data is taken into account to randomize it Make sure not to miss anything.

00:28:03: And one more thing thing called golden path, because it's supposed to be the most popular path for players.

00:28:12: When we want to show some stuff on the way We are making sure that like a biggest percentage of the player will pay this path.

00:28:19: so we're Making sure That This is working.

00:28:22: The priority has

00:28:23: To Be perfect

00:28:24: and all the other things Are

00:28:26: usually the main quest right?

00:28:28: Yes yes yes the Main Path Through the Quest

00:28:31: Dranches.

00:28:32: what then?

00:28:33: We have two golden paths.

00:28:37: If it branches, we have more paths and yeah It's our problem to check all of those.

00:28:41: actually

00:28:42: this branching thing is so common for us lane hour games.

00:28:44: Yeah, we need to be prepared for that hard

00:28:47: right?

00:28:47: I wonder well So how many How many times you finished the game?

00:28:52: do you want a number?

00:28:54: I don't know.

00:28:54: like

00:28:55: many many time we need To finish at many many times because he can see she

00:28:59: counted against

00:29:00: different ways And the truth is, you're not always finishing it.

00:29:05: You can play the prologue for one hundred times before you finish the game because...

00:29:09: First we need to have a game right?

00:29:10: Yeah!

00:29:11: But here I'm also wondering how hard it is for regular developers just playing drafts of the quest For example.

00:29:19: they are super rough It's all grey The characters aren't talking or making some silly moves There no animations.

00:29:25: It isn´t a pleasant experience.

00:29:26: How do you have imagination?

00:29:30: Imagine yourself was the final product rather than just focusing on what doesn't work.

00:29:35: Just basically nothing works at some point, right?

00:29:38: I mean working... It looks horrible and you have a lot of imagination to imagine like okay i'm playing i'm focusing on this narrative structure and then paying attention to all those crazy things happening on the screen.

00:29:52: The truth is when the game in that state or what your are describing we're focussing more not on the final vision because things may change during this time.

00:30:01: So we have to focus on what it is there right now, if the fundamentals are working so that you can make a beautiful later and polish it later... We cannot step into the future with our

00:30:20: mind?

00:30:21: No I just mean….

00:30:21: It's very difficult for us to just focus on those.

00:30:25: On these actual thing which has been evaluated at the end at that point.

00:30:29: But I think it's difficult for everybody on the project to imagine a full game, how is going be with everything coming from every department because you don't know much thing about this point.

00:30:43: once there was time in process of creating a game called Future Complete.

00:30:49: It's called different, it can have different names.

00:30:55: But

00:30:55: after this moment to look at the full game I mean we should be talking about risk before because there is hard to introduce any changes.

00:31:08: But this is the moment when you see a very, very full picture with details and everything.

00:31:13: With all of the audio sounds because post-processing comes up later than the quest thing We don't work like... All themes are working at the same time.

00:31:25: Some things need to prepare something for other themes.

00:31:28: So

00:31:29: it's worth mentioning that This was also the point where we shift our resources from embedded work to, we call it like a global game

00:31:41: where you're right there.

00:31:42: Where we play the game?

00:31:43: We just played a game.

00:31:45: whether these are analysts or testers they play the games and find the findings- This

00:31:50: is moment for paths actually yeah.

00:31:54: so this happens rather.

00:31:55: first of all we need content complete.

00:31:58: It's at more advanced stage in development so advanced production

00:32:03: Having such big knowledge about.

00:32:07: Do you feel like it kind of changed your approach when playing games in a free time?

00:32:14: Unfortunately, yes.

00:32:16: I recently started to play the game just to identify maybe not the box but how these games are designed and what we can learn about them in our games.

00:32:27: so this is... I love that because don't get me wrong if i played those games And this is for my work, but also pleasure.

00:32:36: Sometimes if it's more pleasure I stay the game and try to beat it.

00:32:39: sometimes i play just like ten hours or six hours.

00:32:41: Just have a feeling about the game.

00:32:43: see what we can learn from This.

00:32:47: was that the successes they provided developers To us whether the failures you want avoid in our games.

00:32:52: We have

00:32:54: an analytical approach to it.

00:32:56: You kind of see like what's happening and that

00:32:59: I think people do the same, right?

00:33:00: That's

00:33:01: true.

00:33:01: we know What's under-the-hood at this moment so we know how The game works not only enjoying in We are also enjoying but not Only.

00:33:10: yeah So we're i Think me personally i appreciate more what other creators Are doing But i Know How It's working Under the hood Like, for example when I see that somebody uses a camera in a creatively way In the game and they know how it's made.

00:33:27: It really appreciate that.

00:33:29: I noticed And then we appreciated.

00:33:31: people are doing that you know?

00:33:32: How hard is to make exactly

00:33:34: absolutely Absolutely i think That You have more empathy To The developers Because You Know How Hard Is.

00:33:41: This Is The Miracle.

00:33:42: i Think That Seba Said That in Some Previous Episodes Our Game Director Of Future Force.

00:33:47: That We Make We Make Miracles Right so Yeah.

00:33:50: And it's difficult to make them, right?

00:33:52: Sometimes It doesn't like work well.

00:33:56: so yeah I have a lot of empathy and appreciation for the developers.

00:34:02: Robbie, you said that because I was thinking it's going to be the other way around.

00:34:05: That is gonna be like oh no!

00:34:06: It's very hard to enjoy games now Because i know how everything works and where

00:34:12: they

00:34:12: kind of cut corners.

00:34:14: This could have been better.

00:34:16: Yes

00:34:16: there is this side but... I didn't want to speak about that.

00:34:19: There are these sides when things can get done better If we knew what they were doing.

00:34:25: And then here when there is a scene and you are player at the scene, then the scene ends.

00:34:34: And you see that your spawned somewhere outside like far away.

00:34:39: I'm thinking why can't you be spawned?

00:34:41: At the same point seeing...

00:34:43: Breaking immersion in this case.

00:34:45: Breaking

00:34:45: immersion as beautiful words but yeah it's just the side.

00:34:49: But yeah.. Yeah

00:34:50: i think that There Are Things In Games That I Observe When I Play The Game That Irritates Me because I know that this could be done better.

00:35:00: We have our own perspective, we developed triple A narrative-driven games so it's different than indie right?

00:35:08: And sometimes i'm irritating.

00:35:13: the character at play jump in a weird way but later on playing the game... ...I know doesn't really matter because overall experience is great.

00:35:21: So It does not matter if the character jumps in a wierd way.

00:35:26: So, these are this kind of like experiences.

00:35:29: Like

00:35:30: we have mixed feelings right now basically depends on the game and depends how that the creators done it.

00:35:37: but yet there is something in your brain working differently when you know.

00:35:44: So what's the most challenging thing you would say in terms of like testing RPG system, quest logic or branching narratives?

00:35:51: And also I want to touch somewhere on content lock and kind of getting all those submissions and things done.

00:35:57: Then pushing it out because that always is...I feel like In this studio there are so many things And ship it.

00:36:10: When the content log happens, like day before or night before all these things come... So in morning you open

00:36:18: the

00:36:18: editor and don't recognize that game!

00:36:21: Yes actually there are two different answers.

00:36:24: because what's the hardest thing about testing those massive games?

00:36:30: I think it's to cover everything, because we're just people and also getting tired of stuff.

00:36:36: We work only eight hours a day.

00:36:40: To cover the players that thousands or millions can do is not possible.

00:36:45: so try to cover a percentage like the most popular thing they mentioned.

00:36:52: And make sure there are no problems in their release product.

00:36:59: No, it doesn't like we cannot do.

00:37:01: It's the other thing is that time We would have to have a lot of time To make sure that the game is test like very very properly and we are doing what we can in the time.

00:37:12: And the other thing, the logs.

00:37:15: This is kind of a different thing.

00:37:17: at the logs are part of the process for people that don't know.

00:37:21: there are moments in the process that you need to log this.

00:37:24: submissions too.

00:37:25: make sure that you control what's coming into the build it to control the submits to check all these admins.

00:37:31: It's like I love this metaphor and now i'm using them again.

00:37:34: but we really

00:37:34: like metaphors

00:37:36: Like when we built something on an early stage stop people who want to move the wall.

00:37:47: And this was at the same time being painted by someone else, right?

00:37:52: So this is kind of a metaphor

00:37:54: for putting on it.

00:37:56: so that's why we can never

00:37:58: be done

00:37:59: no walls moving because we paint them.

00:38:01: Right.

00:38:02: yeah We're stopping everything.

00:38:04: just check where are.

00:38:05: how do they build these?

00:38:08: and every thing coming in need?

00:38:12: It's triaged and checked, controlled even before it is implemented into the game.

00:38:18: So yeah there are moments that nobody likes to be honest not people who have stopped.

00:38:26: And we're also like this has a burden on us because everybody is waiting for something.

00:38:34: The bottleneck thing.

00:38:35: Yeah We tried in that too Because we don't want to stop anyone.

00:38:42: We have a counted number of days to do something.

00:38:44: So we had deadlines, so you have two feet in the deadlines?

00:38:48: So it needs to be smart about the logs and they there can be different rules about but depends on that moment in the process depends on which is state-of-the game.

00:39:00: The log can also be only one team not done everybody.

00:39:05: It's different, so there are many details to the process but basically it is about control and making sure that everything is stable.

00:39:10: The game is stable, stabilized... So

00:39:13: when this lock happens a new change needs to be introduced?

00:39:17: Someone doing the change could be small one or whole system... You

00:39:20: never know!

00:39:21: ...is being sent to you so nobody else in the studio actually can download it into the engine.

00:39:28: feel the requirements, does the game... The whole game work.

00:39:32: Does it

00:39:33: break the

00:39:33: whole game?

00:39:34: Yeah exactly!

00:39:35: It doesn't break the editor or the game and paths

00:39:38: etc.,

00:39:38: but once you validate that it enters everybody in the studio can download right.

00:39:42: yeah at the end we have all those sub-mates and the build should be still stable.

00:39:48: being prettier let's say working better.

00:39:51: And this is hard because, you know... Again the bottleneck thing.

00:39:55: So people are waiting for the submits.

00:39:58: It

00:39:58: takes time to check.

00:40:00: We're at a bottleneck and we stop development but do it for good of game

00:40:05: Of course!

00:40:05: This isn't to make anybody...

00:40:08: But yeah..we feel this pressure.

00:40:09: so we need to be lean and effective in what's been done so that we don't stop development Because these costs.

00:40:15: I remember the famous boat from The Witcher III.

00:40:19: It was like, you change something completely unrelated to one thing in a quest or something and then the boat breaks.

00:40:26: This is when we need the QA because if the designer's working on one thing... Exactly!

00:40:31: He testing only this thing in his own environment.

00:40:33: And we are testing it at an environment of game which is very important Because that can affect anything Even with things You didn't think about.

00:40:42: So happens That.

00:40:44: why were there?

00:40:49: So we have bugs, right?

00:40:50: How they are communicated to the team.

00:40:51: What usually is?

00:40:52: what means of communicating all these problems and bugs for development teams?

00:40:58: You said there were many ways of communication.

00:41:03: There's obviously very ad hoc communication.

00:41:08: We use our Slack channel For example Our communicator.

00:41:12: there are reports we prepare and send.

00:41:13: There's JIRA reporting,

00:41:15: right?

00:41:15: So we track backtracking uh... And We use many fields to let people know if this is high priority or severity.

00:41:24: Uh.. They're like assainees themes and stuff so they should follow these as well

00:41:30: Like uh.. Yeah, reporting is one thing but this report needs to catch the attention and explain swiftly What's the problem, right?

00:41:39: If it takes ten minutes to understand the box something is wrong.

00:41:42: You need to improve your reporting.

00:41:44: basically Right

00:41:45: we have some best practices.

00:41:46: that comes with a box To make again things effective.

00:41:49: We're crazy about effectiveness

00:41:51: Yeah but actually this communication Is also very difficult part because... ...we need to be honest and direct But at same time not to be assholes.

00:42:01: Sometimes you look like assholes But that's not our intention.

00:42:05: Of

00:42:06: course!

00:42:06: This is being radically candor, radical candor right?

00:42:10: So you were direct... It was just a small

00:42:12: fix.

00:42:12: hey come on this wall.

00:42:13: one centimeter in the wall

00:42:16: won't change anything

00:42:18: it won't break anything.

00:42:19: we have to be very

00:42:20: assertive in our work because we communicate with many different people teams and characters so different people communicating like in different way.

00:42:29: So the assertiveness is very important because we want to relay a message about data.

00:42:35: We don't want to criticize anybody's work, but unfortunately we are doing that as we're pointing mistakes in their work and not wanting it harshly.

00:42:44: I

00:42:44: think this is like trying to be kind rather than nice.

00:42:47: Right?

00:42:47: We say how it is... ...but do our best for the good of the developers and what's most important – The Good Of The Game.

00:42:54: Brutally

00:42:54: honest!

00:42:55: Always the way

00:42:56: you go….

00:42:57: Yeah,

00:42:57: I'm happy that you mentioned reporting.

00:42:59: Uh...I remember when we are before like hitting a big release We always get reports with the amounts of bugs happening and stability.

00:43:07: And it's always what is the number?

00:43:10: How close are we?

00:43:10: Especially if your work on communication You want to know Before they released how close you were To product which would seem tested as looks perfect That you're ready work on the publishing side of it and the marketing site, communication to PR all that jazz.

00:43:28: That you're seeing also.

00:43:30: how much is actually put in by QA team?

00:43:33: And your seeing like stats going down because we get graphs or everything.

00:43:37: so its a fully blown awesome presentation thats easy to digest.

00:43:42: You are like yeah there's really alot of work going into this.

00:43:45: Also like... In last days.. The last months.... You see whole process which was super interesting From my point of view, for example.

00:43:54: So

00:43:55: yeah data gathering data.

00:43:58: how

00:43:58: many bugs there could be in the program?

00:44:01: Many depends on the face depends if you're thinking about all the bags or maybe box directed to one team.

00:44:09: but they can't beat thousands really

00:44:11: thousands like.

00:44:12: what matters is this should

00:44:13: be close obviously not open thousands.

00:44:16: What are the bugs that really matter?

00:44:18: Of course, we try to avoid all the backs right but will also prioritize and set the severity for the backs.

00:44:24: We tried to avoid those.

00:44:26: mainly dose first of all does that really impact the game in a wrong way?

00:44:32: you can't avoid all of the backs.

00:44:33: no matter how you test The Game No matter what kind of tests You apply game Will always surprise at the end And the players will surprise.

00:44:41: But our aim is to minimize this surprise, right?

00:44:45: So where there's this pipeline?

00:44:46: development pipeline.

00:44:48: We we try to make sure that the outcome of the pipeline it not a big surprise.

00:44:52: What are those levels of severity debugs?

00:44:55: I think It depends how we agree on with production.

00:44:58: Right Depends

00:45:00: what its needed at most.

00:45:01: but they're like three levels The most high one medium and you know was

00:45:05: zero alert Priority zero, so if you don't

00:45:09: know what P-Zero is.

00:45:11: So it's like the highest... It's red lights everywhere!

00:45:15: When you got a p-zero bug then everybody

00:45:18: every hands on deck.

00:45:19: yes yeah for example The game doesn't work.

00:45:22: build this not like

00:45:24: Game breaking

00:45:24: can really kick it off right?

00:45:26: Then as we try to achieve always working games We tried the game that is always working, right?

00:45:36: The build that's able to start.

00:45:41: If it isn't happening as like a blocker... It's P-Zero

00:45:46: And you get an email.

00:45:48: Yes, we have special channels.

00:45:51: We have our

00:45:51: own methods.

00:45:54: let's not talk about it because people are listening and I think good methods for that.

00:45:59: so there is a

00:46:00: game breaking.

00:46:01: they're like important for some visual bugs or functionality breaking then their like cosmetics probably right?

00:46:06: That really depends also on the specialization with him which team can have their own parties inside.

00:46:11: And also the PZR thing changes over development.

00:46:17: It's

00:46:17: different in the later stage of project and then very early stages.

00:46:22: Sometimes you don't pay attention to things that are critical on a later stage when we were at the beginning, right?

00:46:29: I think it is kind... At the begining we do not report everything because there aren't enough things to be reported like all the blocking bugs, ground breaking things.

00:46:39: Then this gate open fluff thing reports every moment When we filter out everything And again We are closing this filter to the most important things because time is running out, unfortunately.

00:46:51: But just say it there's a big chance that if you see a bug after playing or they release in game It was seen and reported.

00:47:02: but for some reasons we didn't have the time of the chances to fix everything.

00:47:09: But we don't have enough fixers, for example.

00:47:12: And that's why the patches after their release... We try to address them afterwards.

00:47:17: So yeah..we're trying cover everything but again we are only just humans.

00:47:22: One more thing about a box like of course P-Zero as p-ones and p-tools.

00:47:25: probably these things need to be addressed immediately.

00:47:28: There is good examples in games.

00:47:30: you can check on the world.

00:47:32: they had boxes but overall there were brilliant because the experience was great.

00:47:37: so be too focused only on bugs and like go mad because of a box.

00:47:44: We need to balance this objective side, player experience feeling with the box.

00:47:49: sometimes it's okay to have little things.

00:47:52: what matters is if the overall experience as good It feels good right?

00:47:57: Yeah I'm happy that you also mentioned post release Because there's always this thing That Like he mentioned You can test things for so much time And still when give game to million players or more they will eventually find something, right?

00:48:11: So how do you manage this?

00:48:14: like?

00:48:14: because sometimes players to some of the weirdest shit you can ever imagine.

00:48:19: You're like I would never think that he'd have a special weird weapon and something together And go instead of going Like The main path around from the back and do something.

00:48:30: Repro steps are very complex.

00:48:32: Yeah.

00:48:33: so how we

00:48:34: reproducing actually what the player did?

00:48:36: based on what?

00:48:37: YouTube videos for example, like Twitch streams or just you're getting the information from players.

00:48:43: Exactly after their release.

00:48:44: they are asking yes so let's say bugs and problems with the game is shifting to customer support.

00:48:52: And customer support, it's like a group that is gathering all of the info.

00:48:56: so if you have any problem reach out to customers.

00:48:59: they will reach us.

00:49:01: They are very active in looking for this because not only them receiving tickets from our players on our customer support portal but also go through forums where Check the social media portals, stuff like that.

00:49:19: And if they report something then let us know right?

00:49:22: Exactly!

00:49:22: They have their own Gira and their own project... ...and are reporting on tickets from all information they could gather.. ..and it's sent to us to try to reproduce, try to debug it, analyze it and find a conclusion.

00:49:36: so this is for other people sending those issues.

00:49:41: The more information you get us, it's the easier for one to find.

00:49:47: So if you have save files or crash files anything that can think of the repro steps before doing this is kind a Q&A job!

00:49:57: To prepare tickets like these.

00:49:59: but the more you tell what they did there will be bigger chances we'll fix them.

00:50:06: If we only had an info about how things work We are not sure why.

00:50:11: What's the reason we can try it?

00:50:12: It works for us and there is this saying in Polish language, ''It''s working for

00:50:17: me.''.

00:50:18: Not working for you!

00:50:20: So yeah...

00:50:22: It worked

00:50:23: on my PC exactly.

00:50:24: Just rest her to your computer.

00:50:26: For the developers actually right some develops are also sometimes reporting bugs.

00:50:29: do you write something I that doesn't work from me but Yeah.

00:50:33: Or

00:50:33: they're recording the bugs for themselves, but we are helping finding a solution and investigating yeah?

00:50:38: There's

00:50:39: always this you know when your editor crashes people tend to send just okay kill it and restarted But its better.

00:50:45: just to summarize what you did.

00:50:47: so it crashed.

00:50:48: So you have more information.

00:50:49: then You can make better

00:50:50: data And easier developers.

00:50:53: The bigger data that we get

00:50:55: As Monika said the more information ,the more context We have the better and easier for us To fix it or address the fix with developers sooner.

00:51:05: Like

00:51:05: with doctors?

00:51:07: Yes, I mean pain!

00:51:08: The game is an organism... This was a metaphor that i used.

00:51:12: this is living organism.

00:51:14: each change in environment even if you are using different headphones we use at work it can break something and change something.

00:51:23: We don't know about it.

00:51:25: so look for solution not seeing anything breaking because of one small.

00:51:31: So, yeah.

00:51:32: The more data the better and we can help.

00:51:35: And

00:51:35: in this post-launch stage I really appreciate that players come to us and tell about their problems.

00:51:41: Of course they want to avoid situations where there is less interaction with players sending us bugs But at the same time it's how you listen to them and improve your game for them Right?

00:51:55: I truly believe our games are as good because of their feedback And we just incorporate changes based on that.

00:52:04: We try, yeah?

00:52:05: Yeah and then...we've always been open with players.

00:52:07: I always liked that because i remember uh..I always tell this story before a joint CDPR was playing which is really at home.

00:52:15: Now had one bug before patch like one point oh eight something Like dad.

00:52:19: it was Lambert would in Kermoran Would fall under the ground.

00:52:23: they could interact With him to progress The main quest and instantly I Was asked To send the log of what happened.

00:52:31: And then I had a community manager reach out to me and say that, okay this is going to be handled in the upcoming update.

00:52:38: told me when the updated it's gonna happen.

00:52:40: up they happened.

00:52:41: thing was fixed on all.

00:52:42: i got all disinformation.

00:52:44: well sometimes have always been position back.

00:52:46: I would have problems within games and either very hard to talk someone that is actually from studio.

00:52:53: uh, ask me for all these things and then have the reassurance that these things are going to be fixed.

00:52:59: So I've always liked that we do that And i also love that We never kind of say Say no like you know in a sense That okay?

00:53:15: until we are satisfied with it and see that players are satisfied.

00:53:18: The resilience of never stopping, I think is super cool!

00:53:23: And mainly thanks to QA right?

00:53:27: Does this happen when nobody can find something?

00:53:35: Mostly if you don't have the data on how it happened... Usually when we have a save file, something like that is easier.

00:53:43: But if you don't know what happened with the steps before it may work for everybody or its about equipment because sometimes the hardware can also do some things.

00:53:55: so... We just need to know stuff and hopefully we can reproduce them.

00:54:02: but there are situations where we cannot.

00:54:05: But we try.

00:54:06: We tried to do it,

00:54:07: must be easier for sure with consoles because everybody's using the more or less same hardware.

00:54:12: but also depends which patch are you on.

00:54:14: some people don't download specific patches so they play like super vanilla versions

00:54:18: and then mods are kind of pain in the ass from us because the mods were breaking the natural original environment game.

00:54:25: So if you have mods and problems.

00:54:29: So, yes.

00:54:30: Yeah that's

00:54:33: why whenever we do updates We tell players to like hey if you have any mods You better get rid of those and then re-install them And see it works.

00:54:41: but once you update To a new version Those most likely won't work.

00:54:47: And then the modders need to kind of update their mods, to fit a new version.

00:54:51: So it's like... The game

00:54:53: is very delicate on its own.

00:54:55: It also could happen that drivers for example are broken because of external software.

00:55:01: Even we have to refresh our drivers from time-to-time when there're new ones.

00:55:06: That

00:55:06: why I always recommend make sure that you have like minimum requirements.

00:55:13: Like all

00:55:13: the requirements are as important if we're not playing on at least minimal ones,

00:55:18: we can't guarantee

00:55:19: exactly what kind of deed and how to do it again?

00:55:21: We can try but there is no guarantee.

00:55:24: If they were not playing by our rules There's No Guarantee That We Can Help.

00:55:31: Yeah

00:55:31: How Do You Guys Keep Morale Going?

00:55:33: Because I Feel Like Your Covering A wide aspect of everything.

00:55:38: And like you said, the game is a living organism.

00:55:40: it's huge.

00:55:42: Before release I know that everybody's stakes are high.

00:55:47: anticipation this incredible.

00:55:49: Everybody just wants to get the game out The door.

00:55:51: there was a lot testing happening.

00:55:53: There has been a lot bug fixing.

00:55:55: How do you keep everybody going?

00:55:57: Because I know that QA is one of the hardest working part of a gaming studio.

00:56:02: And if someone tells me it's not true, i'll tell them It Is True because You Can Tell how Many Things Are Getting Fixed Super Quickly.

00:56:11: But How Do You Keep People Going and Kind Of Keep Morale Going in The Team?

00:56:14: Yeah...I Will Start With This That You Know You're Right.

00:56:19: Sometimes this Morale can Drop faster than anywhere else, because we also see the negative side of the project.

00:56:27: Right?

00:56:27: Of the game... We see these bugs and problems And there are sometimes many of them which is natural.

00:56:31: but if you every day seen a problem You start thinking like they're only problems right?

00:56:39: I think that my approach in this try to show people bigger picture would do it for Beaker thing right for the greater good.

00:56:51: How I say it, right?

00:56:52: For for the good of the game and sometimes The morale boost when you achieve to fix something Right a little think for example the build doesn't kick off It looks very bad but You meet with developers your brainstorm we go through the locks And suddenly yeah works let's celebrate.

00:57:12: I don't say we always celebrate.

00:57:14: We should celebrate more, but i see people really that the spark in their eyes if they think... If they manage to fix or find a way to fix little thing like..if you look at from very high perspective of this Little Thing nothing and peanut right?

00:57:29: But for some people This is a lot And we celebrate this.

00:57:33: and these kind of things probably In my opinion boosted the Morale app I don't know about.

00:57:39: When we are problem solvers, when solve the problems this is a success for us basically as

00:57:44: dopamine hit yeah?

00:57:45: We did

00:57:46: it because obviously It's great to Be enough patient and strong to wait till their release but you know its releases like sometimes years in work entrenched right.

00:57:59: You need to find

00:58:00: after they released.

00:58:01: yes, it's

00:58:02: huge.

00:58:03: If everything is like how do we expect that?

00:58:05: it's probably It's huge not proud of his huge But you need to find a way to, uh, find the satisfaction in the pipeline.

00:58:12: Like when the game is not ready to be launched right?

00:58:16: From the other side I would add that we also like... We gain our boost from appreciation of others.

00:58:24: If there are people at the company who appreciate their job and not doing it basically We feel that we are doing the right work.

00:58:36: If somebody says, I don't want your back you're not correct.

00:58:40: this is working for me.

00:58:41: This is not a bug.

00:58:42: i will do it my own way.

00:58:44: Your ideas or Shit something like That?

00:58:47: This Is Not The Good Thing But Appreciation For What We Do Especially if We're not Decisive People Were Supporters.

00:58:54: So We Stand Shadows.

00:58:55: Let's Say With The Ninjas In The Shadows Making The Game Working And The Quality Working.

00:59:00: And again, when I said working with different characters it's hard.

00:59:04: So sometimes we get different reactions that the better their appreciation once making us working better at a moral boost is no matter

00:59:15: how hard It is.

00:59:16: people as nature of human beings like We want to hear that our job was like Valid right?

00:59:24: Yeah someone

00:59:24: was happy about That.

00:59:25: this little thing doesn't cost too much if you hear dad from time-to-time.

00:59:30: its great it for sure.

00:59:31: support.

00:59:33: Simple

00:59:34: things, little things but like we shouldn't forget about them.

00:59:38: too many times when I've been to any type of demo were doing or hands on.

00:59:44: There are so many times that a QA person would just solve things on the go.

00:59:47: It's like downloading the latest build, it was an hour before we got on stage and still some final checks but I always see QAB as fixers.

00:59:57: They pretty much get stuff done.

00:59:59: they provide you with their builds, test them know its solid then everything works which is incredible.

01:00:08: Two more questions before we wrap up, because QA probably at least in my opinion has all the bloopers of the world.

01:00:15: Like you guys and gals probably collect everything have a million videos.

01:00:20: is that true or false?

01:00:22: It's true, it is.

01:00:25: Since I was working we collect those in our folders and for us obviously to have like a archive of the things that were happening... Trust me

01:00:35: so many players would want access without vaults

01:00:39: because they cannot release

01:00:41: them!

01:00:41: After Witcher III there was this video which actually created from our bloopers outside of our stories.

01:00:52: Yeah, so yes we are collecting that.

01:00:56: obviously the whole backtracker thing is like collecting all the issues but we're collecting bug or your

01:01:07: favorite my

01:01:11: own know because it was so long ago that I was a reporting back side and i work more with the people than with the game unfortunately.

01:01:19: I mean, I love working with the People but they don't have time to work with the Game and report the box.

01:01:23: But asked me team usually ask them where are the bugs?

01:01:27: That you're reporting?

01:01:28: what would do remember?

01:01:30: there is no one bag because there are like categories that I will tell you the funniest one.

01:01:36: So things being stretched among whole level, part of it... You know?

01:01:42: One finger is being stretched amongst a whole game in the universe.

01:01:47: or problems with textures where those blue and red thing... Disco effecting Yeah, face

01:01:58: is completely.

01:01:59: NPCs being bold.

01:02:00: Some more visual ones.

01:02:01: there are the the funniest one's.

01:02:03: obviously all the animations been crooked and stuff like that.

01:02:06: wonky yeah being wonky.

01:02:08: There is nothing that I can say Like this.

01:02:10: one thing was the most funny as well because we have thousands of them but for example they're things those were the visual once better.

01:02:18: We also use sometimes weird naming convention.

01:02:24: For example, there was a bug when the lamp was called pillar of light.

01:02:28: So it's also hard to find those issues.

01:02:34: It is pillar-of-light but... It creates

01:02:36: a pillar of lights?

01:02:36: Yeah!

01:02:38: Harder to find.

01:02:39: There are also bugs that we did for cyberpunk connected with sex scenes because realistic, so and there are different sex scenes.

01:02:49: So they're bugs explaining what the sex scene should look like in a real version.

01:02:57: There's like a lot of stuff about the physics of the body for example as well.

01:03:04: I don't remember the bag, but i can say that was very hardly an advocate for Takamura romance in cyberpunk.

01:03:10: There's much feedback from me about it not a bug.

01:03:14: so yeah this is something they'll remember and still waiting.

01:03:17: maybe it will happen.

01:03:19: yes

01:03:19: there are a lot of people who want to have vacating for that you

01:03:22: wanted too.

01:03:23: make sure what they get before that.

01:03:27: sorry

01:03:28: guys my favorite category funny backs is like physics.

01:03:34: When we learn, there are new rules of physics in the world right?

01:03:37: Destructive tests.

01:03:39: Yes

01:03:40: so levitating characters stuff like that It's good.

01:03:44: if you see it only during development but sometimes after developing

01:03:49: Not

01:03:50: in our games.

01:03:51: That's a funny

01:03:52: thing.

01:03:55: What about?

01:03:56: for the last question, what about people wanting to get into QA?

01:03:59: Like what set of skills do you need have?

01:04:01: because I feel like it needs cover almost everything.

01:04:04: And could also talk kind of QA being... The first step that can take And then from there, go to different departments within a company and kind of work in different parts of development.

01:04:19: I was

01:04:19: waiting for this question

01:04:20: so

01:04:22: we can take it!

01:04:23: Let's answer together.

01:04:24: if i could start... I think that uh.. There is this concept of QA being the entry level of our game dev which is true In my opinion Which is great but Of course We try To make sure That people stay in queue If they don't stay in QA, it's also okay because I really love to see designers artists or producers who are coming from qa Because they already have this foundation.

01:04:52: It's easier to work with us if They have this background Yeah and then they appreciate that they really know what it takes To be QA.

01:05:02: And i think This is one of the misconceptions That not everyone could Be a QA you need to Have like A certain Like approach to

01:05:10: attention, investigative like pattern recognition stuff.

01:05:14: This is patience right?

01:05:16: Patience

01:05:17: a lot of people.

01:05:18: resilience

01:05:20: yeah all those finding

01:05:21: the way how to communicate and confront.

01:05:25: your question was about what skills

01:05:28: you kind of need.

01:05:29: so

01:05:31: Of course there's this set like standard skills like Attention To Details being eager to investigate, having the analytic mindset.

01:05:40: But I think that what else or maybe key nowadays is to be able to confront and question... Or maybe question-and-confront because QA's about questioning things And when you question a thing You need to confront yourself with other people.

01:06:00: It always difficult for everyone no matter if your are QA producer whatever You need to say that.

01:06:06: no, I think it's different because facts.

01:06:08: Right?

01:06:09: So be able to get prepared with the facts and facts can support also your subjective opinion later on right.

01:06:17: so start with that And i think That It Also Touches Something That I love To Say To The People I Work With This Is The Attitude Right.

01:06:27: The Attitudes Are The Multiplayer Of All The Skills And Knowledge you Have If You have The Right Mindset And you are just easy to work with, so that's great.

01:06:40: This attitude is also being proactive and confident.

01:06:51: We agree on something.

01:06:53: we do it no matter what.

01:06:54: sometimes its difficult right?

01:07:01: uh portion of you is like our values then and maybe we can remind them about our our values at this point right.

01:07:11: so be ambitious, of course.

01:07:13: um set the goal in per severe i feel like in school.

01:07:16: yeah

01:07:18: be

01:07:20: honest because we work on that data.

01:07:22: Be kind and respectful to all around

01:07:25: And the gamers think?

01:07:26: Always

01:07:28: remember about the gamers.

01:07:29: Because

01:07:29: QA should be a gamer, so people working in QA Should be playing games and different genre of games not only focusing on one.

01:07:37: but we need to compare.

01:07:38: We need me

01:07:40: able to support this and show it in your like everyday action.

01:07:45: Come come to us

01:07:47: Because you can learn all the engine stuff.

01:07:52: You know, debugging testing different tests kind of test but you cannot learn sub skills

01:07:59: and I think that nowadays especially when we are in the environment as our company works on multiple projects We grow in scale.

01:08:09: QA will also growing scale.

01:08:11: But what's really important is to be flexible and adaptive.

01:08:14: And everyone has to be open for learning new stuff, switching gears like For example now in project X you work as a QA on this and that.

01:08:24: but Be ready that within the New Project which is Like A different project Different Requirements You will have To do Slightly Different Job Right.

01:08:30: So Be Ready for That.

01:08:33: Flexibility.

01:08:34: I think it's Ready

01:08:34: for Change.

01:08:37: Awesome.

01:08:38: This was perfect

01:08:39: Great We know everything

01:08:40: Yeah.

01:08:41: Well, I feel like there's... We could be

01:08:43: talking for hours!

01:08:46: If people want we can do it again with additional questions.

01:08:50: So yeah when you got yours thank-you.

01:08:52: This was really

01:08:53: amazing.

01:08:53: Thank you for inviting us because There is not a lot of QA talks and discussions online.

01:09:00: Hopefully now they will be more

01:09:01: because It's super interesting topic And real heroes in the game dev space.

01:09:08: Anyway, he's a real hero.

01:09:14: Thank you so much for watching today.

01:09:15: we cleared up a lot of misconceptions about the quality assurance team and understood better how this team works from the inside.

01:09:22: It was an incredible talk And I hope you guys enjoyed it.

01:09:25: as always don't forget to

01:09:26: comment like

01:09:27: subscribe all that jazz.

01:09:28: Let us know what your thinking about episodes?

01:09:30: We'll see you off course in next one.