Every time I put a new product into the hands of users for the first time I end up learning something unexpected. Customer research and product development can’t completely predict what customers will do when they get to use your product. Seldom does it go exactly as planned. Things we thought were obvious customers may find confusing or worse, useless. More often, the customer’s use of the product isn’t quite what you expected. The beta for VCTR, Datagrid’s free new web service for tracking server software vulnerability, has been no different. So what have we learned thus far?
First, we had some data clean up to do. Within days of starting the VCTR beta, customers started showing us false positives. VCTR was telling them they had vulnerabilities that weren’t there. Engineering tracked down a couple different sources, but the most interesting was a poor mapping of CVEs to RHSAs. It took a couple weeks but the next beta release due out this week should address the issue.
Second, we learned our API was too complicated. After speaking with a lot of customers during development we thought we understood how customers would script their use of VCTR. However, the first beta customers stumbled over what should be simple queries. Lesson learned. Version 3 of the API is running now and has a much simpler API structure.
Finally, customers provided us with some new use cases. As an example, one customer asked if VCTR could validate his configurations against select repositories. He’d experienced failures due to folks not sticking to the prescribed sources. Since VCTR was already collecting all the needed data we simply added an input for blacklisted repositories. This feature is in the version of VCTR running now.
As the number of users grows we’re certain to learn more.