To Citan - Comments about player feedback

Quote:

---

Originally Posted by Citan

Thanks for the feedback. Some of these are bugs, such as the lack of higher-level meat sources. Others are things I just need more feedback on to make work better.

The other thing to remember here is that the complexities of the game mean there's lots of manual work involved. For example, I've tried to manually adjust recipes that use cheese-based ingredients to have something else nice going for them, to justify that extra work. But it's an entirely manual process. I may have missed some, not done enough, etc.

I guess my point here is that specific details make me happy. I can work with specifics.

---

An issue from the player side is we have no tools or information to collate and report this specific information to you.

You're asking for detailed feedback on your game that's cheap, good, and easy. You're trying to cheat your way around the Project Management Triangle.

Some issues I see:

1) We can't collect any metrics automatically. In order for us to collect information we have to sit at our computers with a pad of paper and a pencil (or equivalents) and write down every action we take (every hit, every bit of damage, every mob we kill, every item those mobs carry, every step of every recipe we craft, etc.)

2) You don't provide specific, detailed information in updates. You make vague statements like "I tried to improve how useful cheese is in recipes" or "I increased drop rates in ZONE-NAME". You deliberately obfuscate the intentions behind your changes by using vague wording. If we don't know what you're specifically trying to do, we can't tell you if you've done it.

3) Information that should be available to us as alpha-testers isn't available to us, because this is an old-school mysterious game and we should discover it all for ourselves. You deliberately hide the numbers from us and expect us to reverse-engineer what you've done.

4) There's a lack of dev maturity and ongoing passive-aggression in responses on known bottlenecks that repeatedly frustrate players.

Some "fixes":

1) Better logging.

2) Better communication on the forums. To preserve the mystery of the game, this could be in a special sub-forum. Start with the "Five W's" and expand from there. Every update should be recorded with details as to expected damage numbers, drop rates, comparisons to other/known systems, xp tables, old and new versions of recipes, level ranges, etc.

3) More information recorded/revealed server-side. When a bug can be pinpointed by making server-side changes that track the nature of the bug, then make those changes. Don't offload tracking those changes onto the community.

4) Tell us what your expected metrics are and define terms for us. If you change a drop rate, how many mobs should we kill before we have a "good" sample size (10? 100? 1000?). Tell us what you think should be profitable for recipes, what the expected DPS is for various skills, which skills are basic and which are advanced, etc. Tell us what you were thinking when you made specific decisions so we can respond to whether your thoughts match up to our experiences.

4) More respectful forum communication. Let us know what can't be changed due to server limitations, what won't be changed due to resource limitations, and what's open to discussion. Trust to shut up and stop "whining" if you explain why something won't happen, instead of jumping to complaining that we're "whiners" for wanting specific things to happen. Drop the passive-aggression and player blaming.



tl;dr - in order to get specific responses from the player base start out by providing specific information from the development team.

Frankly, you'll just have to trust me that we already give you the very best info I have available. I think some of that should be obvious. I mean, in your example about making cheese more valuable, I was summarizing my specific recipe changes. Remember the list of exactly which recipes were changed? That's what I was referring to.

We don't give drop-rate changes in exact percentages because I don't know those percentages. The systems don't work on percents or anything like them. They're complex nested tables with complex weighting systems that can vary per player and per monster. They work well, but they're hard to talk about. So I tell you what I know: something changed by a lot or by a little. That's what we got, so that's what you get.

In general, the answer is always the same: we're doing our best to give you info. If there are places where just a day or three could dramatically improve the info we give you... those days have already been spent. We give out detailed JSON files with a tremendous amount of game data in them. As for testing, I spend much of my time writing automatic validation, because we have no dedicated QA personnel. But not everything can be automated or easily turned into numbers. I know that more info would always be useful, but there's not a lot of big easy-to-digest tables of numbers we're secretly hoarding over here.

This is an ongoing project, and that includes our data and analysis tools. I've already talked about various improvements coming, including more logging, more JSON files, more automated testing that will rule out a lot more bug cases. But right now, right today, this is what we have. Please give the best feedback you can given what you know. That's all we can ask.

Regarding info about game changes, it's more than satisfying for me. It's almost always enough to understand what's going on, and sometimes even too detailed (like the recent food change).

IMO, if there is something that would be extremely useful for us, it would be a (read-only) access to the known issues list, if such thing exists.
. If we see a new bug, we would know for a fact it's new, that devs don't know about it (in this regard, the list doesn't need to be complete).
. If we see a know bug but the list has wrong or incorrect info about it, we could report it in more details or with the correct info.
. If we see a new issue in the list, we can verify that it's real.
. If an issue is removed from the list, we can verify it's fixed for good.

Quote:

---

Originally Posted by Niph

IMO, if there is something that would be extremely useful for us, it would be a (read-only) access to the known issues list, if such thing exists.
. If we see a new bug, we would know for a fact it's new, that devs don't know about it (in this regard, the list doesn't need to be complete).
. If we see a know bug but the list has wrong or incorrect info about it, we could report it in more details or with the correct info.
. If we see a new issue in the list, we can verify that it's real.
. If an issue is removed from the list, we can verify it's fixed for good.

---

Seconded. There was a long post about this on the old forum, but I won't retype it here. The short version is "This is my biggest problem with PG/my most wanted feature."

"1) We can't collect any metrics automatically. In order for us to collect information we have to sit at our computers with a pad of paper and a pencil (or equivalents) and write down every action we take (every hit, every bit of damage, every mob we kill, every item those mobs carry, every step of every recipe we craft, etc.)

2) You don't provide specific, detailed information in updates. You make vague statements like "I tried to improve how useful cheese is in recipes" or "I increased drop rates in ZONE-NAME". You deliberately obfuscate the intentions behind your changes by using vague wording. If we don't know what you're specifically trying to do, we can't tell you if you've done it.

3) Information that should be available to us as alpha-testers isn't available to us, because this is an old-school mysterious game and we should discover it all for ourselves. You deliberately hide the numbers from us and expect us to reverse-engineer what you've done.

4) There's a lack of dev maturity and ongoing passive-aggression in responses on known bottlenecks that repeatedly frustrate players. "


Besides the...heated tone of OP

1. We are not testers in the classic sense of the word. From my limited experience testing another game, a dev gave me access to a test server, I opened a .doc and while playing I wrote down all the bugs I could find. I had limited access to the test server (said test server was up only for a couple of hours a day). Fairly sure this game is actually a free to play perfectly playable game atm. Due to the lack of funds needed to hire actual testers (and probably for marketing purposes too) we get to play for free (and to an extent impact the development process) in exchange to bug reports. Imho a good deal for both sides.

2. I'm okay with vague terms, if I have doubts that a feature is intentional I just report it.

3. I don't understand this point, yes, we already know it's an old school game in which we are supposed to discover things by ourselves (and rate the experience, is it immersive enough? Did we like X zone? Could it be improved?)

4. There is something called "short fuse" but I don't think Citan actually has it (speaking as someone with a very short fuse) More like he works really hard at this and we sometimes complain a lot which most likely tests the limits of his patience. I think calling him "immature" is out of line and I'm surprised he responded to this thread.

Side note-yes, he released the json files (try asking these off any other developer or anything client-related and see if you get anything). Overall this is the only game where I saw this level of interaction between the staff and the players, I get actual responses to my bug reports and the suggestions I make are read and taken into consideration. I doubt my experience is different from any other player's.

Better logging and unit-testing for system output (like drops or recipe pricing) is always a good idea to put in ^-^ Making UI for these things is sometimes a pain, but they reduce workload later.