The Importance of Evaluating Products & Techniques for Yourself

By Coach John Hughes

How do you decide if a particular sports nutrition product, piece of equipment, training method, etc., is right for you, or actually works for you?

If you’re like most roadies, you probably first hear about a product or training method of interest from a riding buddy, or from a review or article in RBR Newsletter or another site, or you may see an ad featuring a pro endorsing a product or device.

Do you just dive right in? Or do you stop for a moment to think about the source of the information or review, or how a certain study was conducted? Does the source or writer have a dog in the race, so to speak? (A financial incentive for touting a product or training regimen?) Was it a true test of a product over a reasonable amount of time and in a variety of riding conditions? (Or one of those “put in on the bike and a take a picture” snap reviews that are all too common?) Was the study conducted appropriately (a big enough sample size, following scientific protocol, etc.)? Were the testers riders similar to you? (Or hard-core racers?)

Finally, after you’ve considered the value of the information at hand and decided to try something for yourself, how to go about it? Do you have a method for testing things yourself? Or do you just wing it and see how it goes?

I often emphasize that we are each “an experiment of one” – which is to say that just because a product or training technique works great for one rider doesn’t mean it will work equally well for the next. You’ve got to evaluate things for yourself, by experimenting on yourself.

Here’s some information to keep in mind about this, and a couple of tips on how to make the most of your personal experiments.

A Recommendation or Review

Many advertisements and product websites include endorsements from one or more athletes. Should you trust one of these endorsements? Does the athlete do riding similar to what you do? If a pro racer recommends a product and you are a health and fitness rider, then the recommendation may or may note apply to you. Is the athlete sponsored by the company, i.e., given free product or financial assistance so that he or she is expected to promote the product?

Does the person making the recommendation have relevant education and training? For example, if someone touts the nutritional and performance benefits of the Go Faster energy bar, does that person have a college degree? Is the person a registered dietitian (RD) and/or board certified specialist in sports dietetics (CSSD)?

Does the person making the recommendation have relevant experience? I.e., a trained bike fit specialist or a shop mechanic probably knows more about a specific type of product than the guy you just met on a group ride talking your ear off about Product X. Your riding buddies, on the other hand, assuming they’re long-time, trusted and experienced riders, are often good sources of product information – as long as it’s based on their actual use, and not on what they’ve just read about.

The RBR contributors are also very good sources of information. We are professionally trained and/or have decades of experience, do much of the same types of riding as our readers, and try to put the products we test through real world riding over weeks and months. The same goes for the training recommendations we make: they come from real world experience coaching recreational and competitive riders over years and decades.

Still, don’t just buy or do what we say! Test everything for yourself!

A Scientific Study

If you’re evaluating a product or technique or program backed by a scientific study, start by evaluating the experimental method. There are two types of valid methods:

1. Double-blind experiment. Has there been a double-blind experiment testing the product? I.e., the subjects of the experiment are divided into two groups: an experimental group that uses the product, and a control group that uses a placebo. Double-blind means that neither the subjects nor the people administering the experiment knows which subjects are using the product and which are using the placebo.

2. Controlled sequential experiment. In some cases testing something in a double-blind experiment isn’t possible, for example, a training method. There’s no way to disguise the training method so that the experimental and control groups don’t know who’s doing what. In cases like these often the test subjects are again divided into two groups, and each group’s performance is tested, for example, by a time trial on an ergometer on the road. The experimental group then tries the new training method for a period of time. The control group continues training their normal way. After the experimental period, both groups are retested to see if the experimental training method resulted in a measurable increase in performance compared to the control group.

When you read about a product or method that claims to have been scientifically studied, ask yourself:

• Whether one of the two above methods was used?
• Whether the study has been published in a peer-reviewed journal, i.e., other scientists have reviewed the experiment to determine if the results are valid?
• Whether the number of subjects in the experiment is large enough so that the results are meaningful. The results from an experiment with only 10 subjects may arise from random chance. The results from an experiment with 50 subjects are more likely to be due to the product or training method.
• Are the subjects of the experiment similar to you? The same activity (cycling, not running)? The same age? The same gender? The same training history? College students often earn money by volunteering for experiments; unless you’re in your 20s, the results may not be applicable to you.

Experiment of One – An Example

RBR Premium Member Steve Koester asked two excellent questions recently, around which I based my last two Newsletter articles:

1. What should he do the last two weeks before his centuries in late June and early July to be “on form”? This applies to any roadie trying for a good performance in any specific event.

2. How should he train over the summer to do multiple different events, cycling and backpacking? After the second century how should he prepare for the backpacking trip? After his backpacking trip in early August how should he prepare for a century a week later? This applies to any roadie who takes a break from cycling, for example, a family vacation or a darn business trip. How do you regain the fitness you had before the cycling break to do your normal riding?

I answered the first question in How to Be “On Form” for a Big Event. I answered the second question in Maintaining Form Despite Time Off the Bike.

After his first century back in June Steve reported, “It went pretty well, and likely better than if I would have been over-trained, although it was difficult to back off on exercise so much. I averaged 15 mph, which is good for me.” He followed my taper recommendation and didn’t overdo it in the last weeks before his event.

After the backpacking trip earlier this month, here’s what Steve did the weekend before his next century: “I rode 65 miles (too much, I know) because I wanted to get my cycling legs moving and also wanted to test a new saddle after getting a bike fit a few weeks ago. [Emphasis added] The saddle was uncomfortable, and it was a long ride.”

There are two important considerations here. First, yes, this was too long a ride for a taper. However, Steve did do something important: He tested whether the new saddle and bike fit worked for him. It was important that Steve do a long test ride like this — a saddle or bike fit might feel fine on a short ride but become troublesome on a longer ride.

Second, I wouldn’t recommend testing anything new just a week before an event. However, a fundamental rule is nothing new on an event. It was more important that Steve test the saddle and bike fit just a week before the event than that he use them untested on the event.

Just before the century, Steve wrote, “Looking at my week, I likely am overtraining but in the past I have always built up to big events, rather than tapered, and it is a tough pattern to break, since I like to get some exercise every day.” [Emphasis added]

Then, after the century, Steve emailed me, “Today I rode a metric century and felt better physically than for any ride this summer. I averaged 15.5 mph, which is good for me and I think could have done a full century and still felt OK. I would like to attribute it to the hard ride I did six days previous but it is most likely because I really laid low the last couple days and did less than usual.” [Emphasis added]

Back in June, he experimented with the taper that I recommended, and it worked for him. Then, in August, he modified what he’d learned in that experiment to fit his specific need — testing equipment before another big ride. He put the experiment of one methodology to work for himself.

How to Do Your Own Experiment of One

Suppose you want to test a new piece of equipment. Or a training method that I’ve recommended. Or a sports nutrition product that’s gotten good reviews in RBR Newsletter.

Set up a Controlled Sequential Experiment as described above.

Start with test #1 to measure your performance before you test the new product or training method. The test should be appropriate to what you’re trying to measure. If you’re an endurance rider and want to test something – for example, a saddle like Steve did – then test #1 should be an endurance ride similar to your normal endurance rides.

It should be the typical length of your rides on a similar course (hilly, flat) and with a similar group of riders (fast-paced, touring pace or solo). You should use your normal equipment, do your normal training and eat and drink what you normally consume.

If you want to test whether something improves your power and speed, then test #1 should be a time trial or a hill climb.

Then make the change to the new product: put on the new saddle or train using the new training method.

Do test #2: to test the saddle, ride the same course (or something similar) with the same group of riders eating and drinking what you normally drink. To test the training, repeat your time trial (test #2). You should eliminate as many variables as possible between test #1 and test #2. Repeat the time trial on the identical course under the same conditions (heat, wind) at the same time of day.

Testing Helps Eliminate Showstoppers

A showstopper is anything that causes you to interrupt your season or stop an important ride before you’re ready to stop the ride. Steve was smart and tested his new saddle and bike fit to eliminate a potential showstopper – the new saddle, which proved to be uncomfortable and could have forced him to abandon his next century.

Similarly, a roadie could develop an injury because of inadequate training. Or a roadie could have digestive problems because the rider consumes food or drink during an event that he or she hasn’t used before. Part of preparing for any event should be repeated experiments to eliminate the potential showstoppers.

If possible, you should stop experimenting long enough before an event to ensure that you have dialed in all the equipment, products, ride plan, etc. – all aspects of your ride – in order to train with those in place for the final few weeks.

---

Coach John Hughes earned coaching certifications from USA Cycling and the National Strength and Conditioning Association. John’s cycling career includes course records in the Boston-Montreal-Boston 1200-km randonnée and the Furnace Creek 508, a Race Across AMerica (RAAM) qualifier. He has ridden solo RAAM twice and is a 5-time finisher of the 1200-km Paris-Brest-Paris. He has written over 40 eBooks and eArticles on cycling training and nutrition, available in RBR’s eBookstore at Coach John Hughes. Click to read John’s full bio.