Project Summary The AWS Pricing Calculator is a tool to help current and potential customers estimate the price of AWS. From a potential customer who is just starting their evaluation to a seasoned professional who has deep understanding of AWS, estimates can be as simple as a single EC2 instance or as complex as dozens of services configured in multiple regions.
Goal The purpose of this study is to expose any fatal flaws to the revamped product design prior to MVP launch. Our research will determine how effective the re-designed tool to help our customers:
Quickly build estimates
Understand pricing & calculations
Add, delete, edit, & clone services
Export/share estimates
Observe User experience with unbiased re-designed features
Discoverability & understanding of service configuration, pricing content, and accessibility of supporting information
Approach: Benchmark I conducted a usability test with our basic and advanced users on the current Calculator to establish the baseline for design improvement. Following the benchmark study, I conducted the same test with our users on the re-designed tool to evaluate its capacity to meet customer needs, address known frustrations, and surpass expectations for ease of use, flexibility, scalability, and speed.
Approach: Measuring Success To measure this improvement, I analyzed the following customer data points from both studies, side by side: task completion, time to completion, delta between the user’s estimate and the ideal (defined by the lowest cost pricing strategy to achieve the architecture solution), and our users’ overall qualitative feedback, including impressions, expectations, and feelings about the two experiences.
Testing Format & Requirements
Session length: 1 hour
Desired number of participants: (Option 1) 10-14; (Option 2) 5-7
Starter Interview: 10 minutes
Testing: 40 minutes, unmoderated
Wrap up interview & SUS survey: 20 minutes
Personnel needs: 1 moderator, 1 observer
Material requirements: Usability study script, observer guide worksheet, SUS survey sheets
Key Personas Solutions architects, engineers, technical leads, or subject matter experts in one or several categories; including support specialist.
Tasks to test
Configure Amazon EC2, EBS and ELB for a specific set of minimum requirements, and to identify the lowest cost option for that configuration. This will gauge users’ ability to complete the task to a level of accuracy, and demonstrate discoverability and understanding of AWS pricing options.
Configure and compare pricing for the configuration between 2 regions. This will gauge users ability to navigate primary feature interactions of the overall calculator chrome (UI), and effectively leverage the tool to make informed buying.
User Scenarios Our user scenarios capture the features and capabilities for which we need to validate usability for MVP launch. These feature explorations and notes for observers will be translated into a more free-form script that allows users to navigate on their own to accomplish the task and provide feedback without designer bias.
