Goal
The main goal of our 1.0 Test Bench update is to improve the consistency of our wand vibrator ratings across different tests by aligning the ratings with our definitions here.
Before this update, the same rating value had different meanings depending on the particular test. For instance, a score of 6.0 on our Max Vibration Power test may have meant we thought the wand vibrator had “poor” strength. But a score of 6.0 on our Use During Sex test may have meant the wand vibrator was “decent” during sex.
Most of the changes in this round of updates are simply minor adjustments to the scores to fit our more clearly defined meanings for each scoring range. We also improved our Rumbliness and Power Range metrics and adjusted the weights of each test in our Overall Score to better reward well-rounded wand vibrators.
Summary of Test Changes
The changes we made in this update are summarized below. You can see detailed information about substantial changes below the summary.
Rumbliness
- Improved the method to normalize frequency ratings to be more robust to new data.
- For the metric designed to reward wands that are both strong and rumbly, we replaced the Geometric mean of normalized power / normalized frequency with the Median of normalized power / normalized frequency
- Updated the final Rumbliness ratings to be consistent with our definitions here
Power Range
- Improved how the number of power levels a wand vibrator has affects its Power Range rating
- Updated the final Power Range ratings to be consistent with our definitions here
Max Vibration Power, Noise, Comfort & Ease of Use, Use During Sex, Build Quality, Compatibility with Mounts
- Updated the ratings to be consistent with our definitions here
Battery Life
- Updated the rating system to penalize wand vibrators that use disposable batteries that you need to keep buying (like AA), even if the runtime is long.
- Also updated the Battery Life ratings to be consistent with our definitions here
Scoring Weights
- Updated how much each test contributes to the overall score so the weights are less concentrated around vibration power and better emphasize well-rounded wand vibrators.
Details of Test Changes
Rumbliness
As mentioned above, we updated how we compute the metric that provides a quick way to see which wand vibrators are both powerful and rumbly. In other words, a wand vibrator that is strong but buzzy should score lower than a wand vibrator that is strong and rumbly. Likewise, wands that are rumbly (low-frequency) but weak should score lower in this metric than a more powerful wand with the same low-frequency vibrations.
Our previous test had 2 issues we wanted to fix.
The idea of the test is to compute the power / frequency of each intensity level, then take the average (mean) across all the wand’s power levels. This gives a metric that penalizes weak or buzzy vibrations. Since our power and frequency measurements use different scales, they need to be normalized to affect the final metric similarly. Without normalizing both components to a similar range of values, a wand vibrator’s power might affect its rating much more than its frequency (or vice versa).
The first issue of our previous rating system was that our normalization method was not robust to new data. We scaled all our frequency values between 0 and 1 based on the minimum and maximum values we obtained for the wand vibrators we tested. However, that means that when testing a new wand, if the new wand vibrator’s frequency was lower than any of our original wand vibrators, the normalized frequency value for every wand would change. So our Rumbliness ratings could constantly change (though only slightly) as we test new wands – even though the wand vibrators themselves had not changed.
To solve this issue, we no longer map the minimum frequency of the wand vibrators we’ve tested to 0 in our normalized range. Instead, we simply map 0 Hz to the 0 bound in our normalized range. This is reasonable since the lowest frequency measurements we obtained are relatively close to 0 Hz. A new wand vibrator will never have a frequency less than 0, so our computed values won’t change when we test a new wand.
Additionally, 1 is the worst-case bound in the normalized frequency range. We used to map the maximum frequency value obtained from our measurements to the normalized value of 1. But this resulted in the same issues we had before. Now, we clip buzzy outliers so that any frequency > 105 Hz (what we consider “extremely buzzy”) is mapped to the worst-case value of 1. This provides a harsher punishment when a wand is extremely buzzy and is more robust when testing new wand vibrators.
Lastly, we no longer take the Geometric Mean of the Normalized Power / Normalized Frequency across all the wand’s power levels. Instead, we take the Median of the Normalized Power / Normalized Frequency across all the wand’s power levels.
Both statistics provide nearly identical results with a similar interpretation. They each reward wands that have a greater percentage of power levels that are both strong and rumbly, and both metrics are less affected by outliers than the arithmetic mean (average). But since our normalized value range contains zeros, we had to ignore zero-values when using the Geometric Mean (the Geometric Mean can only be computed when all values are positive). We do not need to manually exclude certain values in the calculation when using the Median.
Lastly, we updated our final ratings to be consistent with our definitions here.
Power Range
We also improved our Power Range rating. Our old Power Range rating was a weighted average of two subscores:
- Power Range Width subscore: This subscore was computed using the difference between the wand’s maximum and minimum power levels.
- Number of Power Levels: This subscore was based solely on the number of power levels a wand vibrator had.
The issue with the old rating method was that vibrators with a narrow range could still score decently well if they had many power levels. For instance, a vibrator that ranges from 1 out of 10 vibration strength on low, to 2 out of 10 on high has a very narrow power range. So its Power Range rating should be poor.
However, in our old system, if this wand vibrator had 15 power levels, it could still receive a decent rating simply because it had many settings.
Our new way of calculating each wand vibrator’s Power Range score fixes this issue.
Now, we calculate a derating factor based on the number of power levels the wand vibrator has and its power range width. The wand vibrator’s power range width is then adjusted by this derating factor. We use the adjusted width to apply our final Power Range ratings based on our definitions here.
Battery Life
Our updated Battery Life ratings restrict wands with disposable batteries to be ≤ 5.9. In our battery life testing, we run each wand vibrator continuously on high power until it dies. The wand’s runtime is used to award its Battery Life rating.
Consider these two hypothetical wand vibrators:
- A rechargeable wand that lasts 90 minutes on high-power
- A wand vibrator that uses three AA batteries and lasts 90 minutes on high-power
Our previous system did not take the battery type into account. Both these wands would have scored the same in Battery Life. But a 90-minute runtime is a much worse outcome if you need to buy 3 more AA batteries after using the wand. Our new system ensures wands that will cost you extra money in batteries are rated poorly.
Scoring Weights
Lastly, we updated how much each Test category (Power, Noise, Rumbliness, etc.) contributes to the wand vibrator’s Overall Score as shown in the table below:
| Category | Old Weights | New Weights |
| Max Power | 18 | 18 |
| Comfort & Ease of Use | 18 | 15 |
| Power Range | 18 | 14 |
| Use During Sex | 13 | 13 |
| Noise | 10 | 13 |
| Rumbliness | 10 | 11 |
| Build Quality | 8 | 8 |
| Attachment Compatibility | 4 | 4 |
| Battery Life | 0 | 3 |
| Mount Compatibility | 1 | 1 |
We slightly increased the weights for our Battery Life, Noise, and Rumbliness tests, while slightly decreasing the weights of our Comfort & Ease of Use and Power Range tests.
Since our Max Power and Power Range ratings are correlated, we felt that vibration power affected the Overall Score too much in our initial system. Additionally, since Comfort & Ease of Use and Use During Sex are correlated, a wand’s general maneuverability also greatly affected the Overall Score.
While these aspects are important, we felt that Noise, Battery Life, and Rumbliness were underrepresented in the Overall Score. Our updated test weights seek to reward well-rounded wand vibrators better.
