Who Needs Physical Testing Anymore?

Computer models and simulations are playing an ever-growing role in vehicle development. Everyone is talking about this, to the point that it is redundant to elaborate the point further. Yet, this fact raises an interesting question: what is the role of physical (real-world) testing in the age of machine learning, driving simulators, and Everything-In-The-Loop-Simulation?

It is easy to forget—or ignore—the role of physical testing. For example, you can create a CarSim model out of the can with “generic” data for suspension kinematics and compliance properties or tire properties. Just go straight to the juicy ADAS development, no need to slow down and consider the validity of the “generic” properties  for your application. While there are certain applications that allow for approximate models, they may be fewer in number than is commonly assumed.

It’s time to talk a little bit more about the current role of physical testing in vehicle development.

GIGO is Still True

Garbage in, garbage out. This phrase was coined sometime in the last century to describe an inevitable occurrence in computer programing (see Rob Stenson’s article about its nebulous history). GIGO expresses the principle that you get what you put in. If your inputs are bad, your outputs will be bad too. This principle goes back much farther than the invention of room-sized computers. Well before people were punching FORTRAN into index cards, Charles Babbage expressed the same principle about his mechanical difference machine:

“On two occasions I have been asked,— ‘Pray, Mr. Babbage, if you put into the machine wrong figures, will the right answers come out?’ I am not able rightly to apprehend the kind of confusion of ideas that could provoke such a question.”

Babbage was expressing that, contrary to Dilbert, accurate numbers are better than the ones you make up.

GIGO still holds true. It is easy to slip into thinking that as computing has progressed we have moved away from this rule. There are at least two reasons for this. First off, we have reduced the amount of garbage we put in. Stenson attributes GIGO’s waning popularity to the reduction of mistakes such as typos—but this only demonstrates that the rule still holds true. Fewer trashy inputs have led to better things on the output side. The increasing complexity of our models also leads us to neglect the impact of GIGO. No longer are we building machines that can only perform arithmetic. We are now building models that simulate the physical realm as well as the mathematical realm, with all its traffic scenarios, weather patterns, and street signs. But still today, AI must be trained, software must be written, and vehicle models must be built. The outputs can only be as good as the inputs.

The GIGO rule has obvious import to vehicle dynamics modeling. The validity of a vehicle dynamics model (one built in CarSim, CarMaker, or ADAMS for example) is dependent on the inputs used to construct it. If you use garbage parameters and force curves to construct the model, then you will get an animation of a non-existent car driving with garbage performance. But, if you use real-world data to build the model  and validate its performance in the real world, you might just end up with a powerful model representing a real vehicle with real-world performance.

Ultimately, a model is built to model something real. A good vehicle dynamics model represents the actual vehicle because it’s built with the right equations and it’s fed the right data inputs.

Real Matters

Today’s automotive engineers are not merely creating car simulation video games (though some are doing that), they are developing the future of transportation tools. Many of the paradigms around automotive transportation are being questioned and re-designed, even as new technologies are being refined and established in the market. These developments are not located in the fantasy world of the Jetsons, but in reality. This means that the stakes are high. Real people—with very real lives—are riding in and driving these things. Transportation developments must be firmly grounded in how vehicles actually behave and perform. What is real matters. Physical testing plays the crucial role of grounding virtual development in the real world.