I mentioned in my introductory post that I would talk about what we do in support. One of the recurring tasks for me is the creation of the Support Intelligence and Analytics report, a.k.a SIA. We take a good look at all the Service Requests (SRs) that we received during the quarter, and try to turn all that data into actionable information, otherwise know as "knowledge".
We then present that knowledge to the other teams at Autodesk. Development obviously has a big interest, but sales and docs get a good look too. We use the report to articulate the "Voice of the Customer" or VoC (we love acronyms). This is just one way in which we try to become more customer focused, and make sure the releases of Maya have value for you. So what's in the report? I'm afraid I can't share the entire thing, but I'll try to give you some insight, and a bit of a preview. When we log a Service Request, we try to capture all kinds of data that help with solving the SR, like contact information, platform, severity and priority of the issue, where it occurs and what impact it has. If the issue comes from a bug we capture that too. We also look at the involvement of our partners, whether enough information was provide, whether the Product Support Specialist required help from others, etc.
To get an idea of how an issue affects you, we classify the Service Requests using this list, in descending order of just how serious it is.
- Data Loss/ Corruption
- Incorrect Result
- Error Message
- How to
We also use a very long feature list, that is continuously updated to capture where in the software the problem occurs. This feature lists matches the features that we also use in our bug tracking system, so that it's easy for us to talk to the engineers.
When I look at how the SRs are distributed, I see that about 25% of all SRs are How-to's, 20% have are related to configuration, 19% have something to do with Incorrect Results, and 8% are related to crashes. Data Loss is negligibly small.
Looking at the How-to section I find that most of the How-to's are related to scripting in MEL and Python. I notice that the number of Python calls keeps increasing, while MEL is decreasing. That's good news for us, Python adoption is going where we like, but it looks like you need more documentation or examples. I'll have to talk to docs about that, or make the existing material easier to find for you.
The most frequent issues with Configuration have to do with configuring network licenses, one platform stands out; Mac OS X. It looks like Mac administrators find setting up a license server harder than most windows servers. We have it all documented, but few people can be bothered to read the hundreds of pages of documentation. We need a quick start guide for setting up a license server on a Mac, or maybe a video tutorial.
In the Incorrect Result category we find all kinds of things, but rendering issues appear at the top of the list. No surprise, really, that's where it all comes together and is often the first time you begin to notice problems that have been there all along.
Here's where I spend most of my time, trying to piece together what the underlying root cause for the problems is, grouping defects and trying to come up with recommendations for a list of top items that need to be addressed.
One of the things that still puzzles me after 15 years in this business is how few bugs are duplicates. It sometimes looks as if people that encounter bugs are the only ones that run into it. That would be a good thing if it was really true that all reported bugs occur only under very special circumstances, but I somehow doubt that.
I tend to work with the assumption that 1 reported bug means that at least 100 users must have encountered it. It seems that some of our users feel that we "already know about this". Some people have told me "How can you not know about this, it's all over the internet?" Well, we do follow a lot of forums (more about that in an upcoming post), but it's not a safe assumption that we know about every bug mentioned on a user forum. If you did not report a bug yourself, you should assume that we do not know about it, and that it will not be fixed. You would do us, and yourself, a huge favor by sending in the CER reports and going to Help->Report a Problem... or Help->Suggest a Feature... directly from within Maya whenever you encounter a problem.
Other things we add to the report are which solutions from the knowledge base were viewed most often, and from all that data emerges a view of how well Maya is doing? Are we meeting our goals for adoption of FBX? Are people upgrading to 2009, or are they still using older versions? Is 2009 is higher quality release than 2008? Lots of questions, and hopefully, at the end of a few days of number crunching, lot of answers.
I hope you found this informative and that you'll use the comments section below to add suggestions, ask clarifying questions and let me know what you think we should address in upcoming versions.