I love how SOE picks their beta testers. - Page 21

Calthine · 10-11-2006, 05:07 AM

Quote:

Originally Posted by LFG

This entire statement by Gallenite is bullshit and misleading:

http://eqiiforums.station.sony.com/e...=126813#M20104

Sure, at the TIME he wrote that, those guilds were only testing combat changes and not EoF content, but does he actually expect us to believe those guilds will not also test EoF raid content? Nice way to mix words, but we weren't fucking born yesterday.

FYI - there is no travel between EoF and the rest of Norrath atm in Beta.

Ishbu · 10-11-2006, 05:58 AM

I bet there is by, oh i dont know, wednesday.

Ever · 10-11-2006, 11:47 AM

Quote:

Originally Posted by Khalan

Beta for this expansion is much too short imho, their are 7 Tiers of Content, 3 Tradeskill Classes, Raids, Combat Changes and a fuckton of other things to test and POLISH! (key here) and 1.5 months is simply not enough time.

They should honestly push it back a month to Dec 14th.

I completely agree with this statement. Mind you, I have no clue what is or isn't "complete" and ready to "test" but I expected beta signups to start _atleast_ one month prior, and beta testing to last about three months. Based on whats been stated in this thread, it seems otherwise and possibly shorter then DOF or KOS (poor memory, so dunno exactly how long those betas were anymore) but supposely 'much much larger' than either expansion. Personally, I think I'm just going to wait for EOF to be on sale to buy it myself, seeing as I don't have any character slots anyway.

Pinski · 10-11-2006, 12:41 PM

Quote:

Originally Posted by Calthine

FYI - there is no travel between EoF and the rest of Norrath atm in Beta.

Unless you get dev help.

Ddrak · 10-11-2006, 03:34 PM

Quote:

Originally Posted by Kriddle

You know SOE should test all aspects of the game. They should not however pick testers that play 3-4 hours tops a week. They have a list of top questers in the game so pick a few of those to test questing, they have a list of top raiders so pick a few people from several top raiding guilds, they also have a lot of players that do nothing but fight heroic or solo content so find those people and test. They dont need to get people that cancel their account for 3 months and decide to activate their account just to sign up for beta.

SoE should just send out contract positions for testing to a group of people that have demonstrated the in-game ability to deal with certain stuff. In short, pay people that are qualified to look at the content to actually test the content using a primitive legal tool called an "employment contract". Would take maybe a few weeks of 8-10 hour days and it would be done.

If you get people to test for free, with no stipulations on what to do when testing then you're not going to get anything properly tested.

Dd

RadarX · 10-11-2006, 04:09 PM

Quote:

Originally Posted by Ddrak

SoE should just send out contract positions for testing to a group of people that have demonstrated the in-game ability to deal with certain stuff. In short, pay people that are qualified to look at the content to actually test the content using a primitive legal tool called an "employment contract". Would take maybe a few weeks of 8-10 hour days and it would be done.

If you get people to test for free, with no stipulations on what to do when testing then you're not going to get anything properly tested.

Dd

They already have a QA group they pay but this is just too much content. The tons of people help do load testing, and besides you are increasing the cost of the Expansion by hiring extra staff. No company hires beta testers to my knowledge, but I could be wrong.

Taea · 10-11-2006, 04:56 PM

Beta testers aren't the only people looking at and testing the content. They have in house peeps doing that as well. What they need are loads of people doing all kinds of crazy things and trying to break whatever they can - that's what most of the "player" beta testers can do.

Ddrak · 10-11-2006, 05:09 PM

I know they have in-house people testing content, but the in-house people are almost uniformly not high end raiders. If you want to test the high end raiding then pay a bunch of high end raiders to do it for you.

My point is that unpaid and uncontracted player beta testers are under no obligation to actually test anything, and typically they don't test anything more than the most obvious paths through things while trying to get a glimpse at as much stuff as possible. If you're not a software developer then you probably don't fully understand the faulty reasoning behind that but it's something like "I got 1000 people to drive my new car design for a couple of weeks, I didn't check where they drove or even if they did drive, but what could be wrong?".

Load testing? Unlikely. If you've ever been in a beta there's nowhere near enough people online to load test the world. Maybe a few random parts of it, but not the whole world.

The fact is, it's not "testing" by any software development definition of the term. It's effectively throwing a bunch of relatively unqualified people at some build snapshot and praying that someone finds something. Testing is having people follow rigorous plans to ensure every part of the world is functioning the way it should.

Add to the expense? Do the math and you'll find that it actually makes the release cheaper when it produces a more stable end product. One of the big issues with the software industry is testing is often viewed as an overhead rather than a fundamental part of the process that actually is more important than even writing the code itself.

Player "beta testing" is a cheap way to keep the buzz going. It's a pretty horrible way to ensure a stable product.

Dd

Kiara · 10-11-2006, 05:18 PM

Quote:

Originally Posted by Ddrak

I'm sorry... You said something, I know. Do it again, really slowly and I promise I'll pay attention this time * nod *

This has nothing at all to do with trying to get you to talk more for the accent. Honest

RadarX · 10-11-2006, 05:33 PM

Quote:

Originally Posted by Ddrak

I know they have in-house people testing content, but the in-house people are almost uniformly not high end raiders. If you want to test the high end raiding then pay a bunch of high end raiders to do it for you.

My point is that unpaid and uncontracted player beta testers are under no obligation to actually test anything, and typically they don't test anything more than the most obvious paths through things while trying to get a glimpse at as much stuff as possible. If you're not a software developer then you probably don't fully understand the faulty reasoning behind that but it's something like "I got 1000 people to drive my new car design for a couple of weeks, I didn't check where they drove or even if they did drive, but what could be wrong?".

Load testing? Unlikely. If you've ever been in a beta there's nowhere near enough people online to load test the world. Maybe a few random parts of it, but not the whole world.

The fact is, it's not "testing" by any software development definition of the term. It's effectively throwing a bunch of relatively unqualified people at some build snapshot and praying that someone finds something. Testing is having people follow rigorous plans to ensure every part of the world is functioning the way it should.

Add to the expense? Do the math and you'll find that it actually makes the release cheaper when it produces a more stable end product. One of the big issues with the software industry is testing is often viewed as an overhead rather than a fundamental part of the process that actually is more important than even writing the code itself.

Player "beta testing" is a cheap way to keep the buzz going. It's a pretty horrible way to ensure a stable product.

Dd

You didn't state raiding content, you replied to a general content statement. Are we talking about testing raiding or are we talking about testing all the content? The high end raiding, they bring in teh uberest raiderz to tackle that. They dont' need to get paid, their motivation is they get a leg up on everyone else by seeing the content.

Umm yeah I've been in a number of betas and of course they aren't testing the whole world, just zones of the expansion. Along the lines of load testing is making sure there is enough content spawning for a full zone.

I hate to be the one to tell you this....this isn't the next version of Windows or some Accounting software. This is a game. This is how the industry tests games, with a beta. Vanguard, PoTBS, Gods and Heroes, EQ2, EQ1, WoW, SWG, the list goes on. Is it a good idea? Only if the cost doesn't exceed the reward.

While we're on that...do the math? Do what math? We're speaking entirely of theoretical values. How much is one bug worth? Two bugs? 3 major bugs and 2 minor ones? How would it make the release cheaper? They would be able to work on other stuff instead of fixing bugs? Developers get paid whether they are adding new content or fixing bugs, so you aren't seeing anything there. You can't really price "customer sat" either because if they aren't complining about expansion bugs, it'll be something else. You are right, testing is an overhead cost. Feel free to try and change the entire software industry, because I agree with you testing isn't taken serious enough. I don't think hiring people to test is the way to go. Either you have enough QA or you don't.

Well of course beta is about buzz too. SOE needs this product to succeed, and has a lot riding on it. I can't blame them for using effective marketing. People are excited about EoF stuff. I've seen all kinds of traffic increases to fan sites because of EoF.