How We Test Vacuums
See How We Test Other Products:
A vacuum cleaner is only as good as how much dirt, debris, and pet hair it can pick up. At Reviewed, we put vacuums through a series of rigorous tests so we can tell you how well a vacuum will perform in your home. That way, we can recommend cleaners that work in real-world scenarios, and you can decide which cleaner is right for you.
In order to help our readers make an informed purchasing decision, our vacuum cleaner rankings are based both on objective test performance and subjective experiential assessments. To see a detailed break down of our testing methodology, read on.
To optimize vacuum cleaner performance, we keep our vacuum lab space at a positive air pressure via air blowers that ensure the constant removal of extra particles floating in the air. This way, the vacuum cleaner does not have to contend with reduced airflow (and consequently, reduced suction) due to particulates in the air.
Preliminary Observations and Calibration
After unboxing and assembling the vacuum, we weigh it. This total weight (in pounds) includes any and all attachments that are meant to reside on the vacuum cleaner itself. Because consumers have to push and pull the vacuum cleaners, carry them up stairs, and put them away after use, lighter vacuums will have a higher weight score than heavier products.
Additionally, we also weigh the empty dirt receptacle (i.e. the bag) so that we have a baseline from which to quantify the dirt removal that is part of the cleaning performance scores.
To ensure that the important parts of the vacuum cleaner are working properly, we plug it in and turn it on, and leave it running in its upright position for 15 minutes. Assuming no problems appear, we proceed with the objective testing.
The bulk of our objective testing deals with the vacuum cleaner’s ability to remove dirt and debris from a variety of open floor surfaces. However, because people don’t vacuum in... a vacuum, we also take a look at the product’s edge cleaning, furniture clearance, mobility, noise level, and a few other spec-based aspects.
You can’t test a vacuum cleaner’s performance without first making a mess. We want to know how effective a given vacuum is at picking up messes both large and small, so we test the vacuum cleaners with two distinctly different types of messes.
The first type of mess is mostly comprised of fine-grained sand that is a good stand-in for the stuff that accumulates in your carpets and on your floors from daily use. The second type of mess is made up of larger items, like uncooked pasta, that is representative of common household debris in weight and shape.
In order to ensure fair testing between products, we make our dirt and debris the same way each time, buying the same kind of pasta and sifting the same style of playground sand ourselves.
The “dirt” is mostly playground silica sand that has been sieved to various grain sizes. We also add in a bit of talcum powder, for a total dirt amount of 100 grams. This formula matches the ASTM standard for vacuum cleaner testing.
- 0.9 grams of silica sand sieved to size 30/40
- 31.5 grams of silica sand sieved to size 40/50
- 41.4 grams of silica sand sieved to size 50/70
- 13.5 grams of silica sand sieved to size 70/100
- 2.7 grams of silica sand sieved to size 100/140
- 10 grams of talcum powder
These sand grains and talcum powder are combined and stirred by hand until it is evenly mixed.
The “debris” formula consists of much more common items: elbow pasta, uncooked rice, and pet hair. These items vary in size, weight, and ability to cling to a carpet, giving us a good idea on how a given vacuum cleaner can deal with these kinds of common day-to-day messes.
- 40 grains of uncooked rice
- 20 pieces of uncooked elbow macaroni
- 6 tufts of pet hair, each 1 square inch in size
Our three test surfaces include high-pile carpet, low-pile carpet, and linoleum. The dimensions of the actual testing area on these three surfaces are 54” long and 18” wide.
To ensure accurate assessment of the cleaning performance, we remove remnant dirt or debris between each test run, with a vacuum other than the one being tested. We test the two carpet surfaces with both the dirt and debris messes, while we only assess the linoleum based on its cleaning of the debris mess. Each mess test is performed twice on each surface.
For the dirt test, we evenly distribute the 100 g of dirt throughout the designated test area in a tight “S” (or rapid back-and-forth) hand motion. Once the area has been covered in dirt, we run an empty 28 gallon Brinly lawn roller across the test area three times to thoroughly embed the dirt into the carpet—the same way your feet would at home.
Before each test, we engage any manufacturer-recommended settings for the kind of flooring we’re cleaning. We then push the vacuum across the dirty test area for as many passes as it takes to cover the entire test area. Immediately after the test has concluded, we weigh the dirt canister in order to measure how much of the 100 g of dirt was successfully removed.
This is a departure from many standard vacuum cleaner tests, in which the user cleans the same spot with up to 15 passes. We consider that an unrealistic representation of how most people clean, so we reduced the number of passes: Depending on the width of the vacuum cleaner head, it typically takes about three passes (a forward-backwards-forwards motion) for a regular, upright vacuum cleaner to clean a test surface.
For the debris test, we evenly sprinkle the macaroni and rice mixture across the test area, with the six tufts of pet hair occupying each corner and the middle of two of the long sides of the testing area. We clean the test area the same way we did with the dirt test. However, because debris gets destroyed by a vacuum during cleaning, we count the pet hair tufts, macaroni pieces, and rice grains left behind on the carpet to determine how well the vacuum cleaner was able to remove the larger debris.
Vacuum cleaners that consistently remove a high amount of dirt and debris from the three test surfaces will have high cleaning performance scores, while vacuum cleaners that fail to pick up large amounts of dirt and/or debris will have low cleaning performance scores.
Dirt tends to collect where the floor meets the wall, which just happens to be a tricky spot for most vacuums to reach. To assess how well a vacuum cleaner can clean at the perimeter of a room, we created a model of a corner baseboard (two pieces of wood joined at a right angle). To test it, we place the model on the high-pile carpet area and sprinkle talcum powder in a swath about two inches wide along the edge of the two pieces of wood. The vacuum cleaner is placed right up against the wood, and is pushed, in a single pass, along the wood until it encounters the other board. This allows us to test front and side edge cleaning.
We then measure how much talcum powder remains. Removing a wider swath of talcum powder from both the side edge and the front edge will result in a higher edge cleaning score.
We know that dust and pet hair hides underneath the furniture. To determine just how easy it is to clean beneath a sofa, bed, or table, we make the vacuum participate in a limbo contest.
Our limbo apparatus is a wooden dowel suspended between two pieces of wood. The wood has holes bored into it at heights of 2”, 4”, 6”, 8”, 10”, and 12”. Starting at 12”, we place the apparatus on the high-pile carpet and then see if and how far the vacuum’s brush head can reach beyond the dowel.
A greater reach beyond the dowel for smaller heights above the carpet surface makes for better cleaning beneath more furniture pieces, and will result in a higher furniture clearance score.
Pushing a vacuum cleaner means that you have to exert enough force to overcome both the weight of the vacuum cleaner and the vacuum cleaner’s suction to the floor.
This is especially true for high-pile carpet, which is where we perform our mobility test. We place the vacuum cleaner on the carpet and attach a scale to the vacuum handle to see how much push or pull is required to move the vacuum forwards or backwards, respectively.
Lower numbers means that it takes less effort to move the vacuum cleaner across a carpet, and will result in a higher mobility score. Higher numbers mean that you really have to put your back into moving the vacuum across the floor, and will result in a lower mobility score.
As much as vacuuming is occasionally necessary, people avoid vacuuming for two reasons: firstly, it’s boring, and secondly, it’s loud. It’s hard to binge-watch your favorite shows on Netflix when your roommate decides to vacuum the living room.
A quieter vacuum is obviously better. To determine just how quiet the vacuums we test are, we first measure the ambient noise level of our test room with no vacuuming noise in A-weighted decibels (dBA) in order to establish a baseline sound level, and then repeat the process with the vacuum running—brushroll and all.
To do this, we turn our backs to the vacuum cleaner, walk about three feet away, and measure the sound level on a sound meter held at about chest height. We then do the same thing while the vacuum cleaner is turned on, and report that dBA value. The louder the vacuum cleaner, the lower the noise level score.
Capacity, cord length, and attachments
Whether it’s a bag or a canister, we calculate the volume of the dirt receptacle, and measure the length of the vacuum cleaner cord. Larger canister volumes means less frequent emptying, and longer cords mean that you can clean more floor area without moving to a new wall plug; these attributes will result in higher capacity and cord scores. Conversely, a smaller canister means a reduced cleaning capacity, and a shorter cord is inconvenient for cleaning large rooms; both of these attributes will result in lower capacity and cord scores.
We also count the number of attachments included in the product packaging. More attachments translate into increased flexibility when it comes to cleaning (i.e. a short attachment for stairs, a narrow attachment for under appliances, etc.), and will result in a higher attachment score.
While the cleaning performance is the most important thing about a vacuum cleaner, you still have to physically interact with the vacuum cleaner. With our subjective assessments and anecdotal evidence, we can give our readers the scoop on the full vacuuming experience.
After all of the objective testing and spec-based calculations have been completed, we have enough experience with the product to then evaluate the vacuum cleaner from a more personal perspective: how easy was it to remove the filter and the canister? Was it easy to store? Was the cord management system hilariously complicated? Were all those attachments useful, and easy to store?
In aid of providing our readers with a well-rounded vacuum cleaner review, we answer these questions, and more.
With our combination of rigorous objective testing and experiential assessment, we can make solid recommendations about vacuums to buy (or avoid). To stay up to date with our most recently tested vacuums, check out our growing library of vacuum cleaner reviews. If you want find out about the best performing vacuum cleaners, check out some of our vacuum cleaner Best Right Now articles, like the Best Cordless Vacuums, or the Best $400 Vacuum Cleaners.