Recently, there has been a drive to integrate artificial intelligence and IoT into undergraduate engineering curriculum in an immersive and exploratory way.
The University of Waterloo recognized this and naturally sought integration of the latest edge computing devices such as Azure Percept in a pragmatic way, benefitted by Percept’s zero-to-low-code platform, which incorporates vision and audio in a diverse prototyping ecosystem.
Integrating this type of device into an academic undergraduate design activity is one of the main missions of the Engineering IDEAs Clinic™ which supplements a traditional engineering curriculum with open-ended activities designed to spark student self-learning and exploration. Through Ideas Clinic activities, students will learn good engineering practices by experimenting with real-world engineering problems; they will gain an appreciation for the connections between courses in their curriculum; they will apply their analysis skills to complex, open-ended problems; and they will exercise their creativity, judgement and problem-solving skills while acquiring soft skills such as communication and teamwork.
Most importantly, the activities will provide students with a breadth of hands-on experiences in an enjoyable and safe environment. Through these activities, students will engage in memorable and practical learning experiences that enhance their abilities to become innovators in industry and leaders in experimental research.
Structural Experiment in the Loop
One of the recent activities developed in the Ideas Clinic is a structural “experiment-in-the-loop” where important data is collected and analyzed in a feedback optimization loop to aid with accelerated product design. More specifically a structural experimental rig was created and instrumented using a combination of both simple and research grade equipment as well as the Azure Percept device.
The application here is to control and analyze a structural loading experiment that is dynamic in nature. A simple commercially available cantilever chair has an adjustable and programmable load actuator attached to it which simulates real world usage from the product in the field (Figure 1a).
Using a variety of sensing technologies, key performance indicators are determined and tracked to understand the structural deficiencies in the device and enhance the product optimization process using simulation tools such as finite element analysis (Figure 1b).
The methodology employed to accomplish this accelerated product design was that of an inverse design approach. Our objective is to identify and understand the real-world load cases and design constraints of the product, while directly identifying the structures and configurations whose functionality comes close to the desired target.
Consequently, using image analysis to understand the deformed state of the chair under load is where the Azure Percept device has enormous potential. The camera-enabled system on module (SoM) provides an extra dimension of instrumenting such an experimental setup as motion tracking of various critical locations can help with better understanding of the structural response.
Image Capture
The simplified schedule of the experiment in the loop workflow is that a representative structural durability block is applied to the instrumented chair that simulates some portion of the life of the physical chair out in the wild.
The load actuator receives feedback signals from a variety of sensors to control its motion to simulate the various load cases that were obtained from the real-world loading conditions that were captured earlier in the process.
Figure 2 illustrates one of the primary signals (load feedback) used to control the experiment in the loop as a function of cycle number (in more simplistic terms, device usage intensity vs. usage duration). This was characterized for a number of different types of structural loading (5 types shown in Figure 2).
Each of these types of loading have either different loading intensity, a different load direction, or both, which using the theory of superposition could allow interpolation of all load cases which are combinations of each through the superimposed response of each case. Images of an LED array affixed to various parts of the structure were captured using the Eye SoM and used for subsequent image analysis to track the motion of key features in an absolute sense (displacement tracking), as well as relative to one another (mode shape detection), to determine and correlate the intensity and type of loading that is being observed throughout the experiment. This is then used as feedback to the experimental control to replicate conditions that lead to accelerated wear of the structure to highlight product deficiencies and areas for future optimization.
Many sets of this type of data were collected for several chairs, to better understand the statistical variation inherent in both the process and the material/structural properties of the chair itself. This repository of data was then analyzed both individually and in aggregate to aid in informed design decisions and accelerated product optimization to improve performance and reduce systemic failure modes.
Image Analysis
The image analysis step is where the optical data obtained from the eye SoM is and then is tracked and registered using a cross correlation technique known as digital image correlation (DIC) which allows for accurate 2D measurements of images, and more usefully, tracking these measurements over time. If a number of datapoints is collected and analyzed to obtain its position, tracking them over time enables one to calculate the “strain” (or change in length) which effectively describes how intense the elongation is within the material which leads to structural failure. Figure 3 illustrates a contour plot, which is generated from an image captured at “peak load” by calculating the engineering strain of several hundred integration points (facets) along the surface to get the entire 2-Dimensional field of interest. This output information is crucial in the next step in the process, which is product optimization where it will be used to enhance the design and limit premature failures.
Product Optimization
Utilizing this response data collected from the physical experiment and then evaluated using the image analysis algorithm, intelligent data driven decisions can be made on how to improve the inherent deficiencies in the performance as well as the overall product design. Furthermore, these physical datasets are instrumental in acting as an absolute calibration mechanism for the entirely virtual simulation model that is done in parallel via finite element analysis.
Figure 4 illustrates the process flow of information required for the accelerated product design approach where physical testing yields valuable datasets from a variety of sensors, most importantly those from the camera SoM. Identification of the critical locations which are imaged experimentally are driven by preliminary finite element simulation results with low predictive accuracy, which are later improved to a high level of absolute predicted accuracy by the experimental data acting to correlate the model parameters.
Generally, this iterative process loop is what is time intensive in traditional product design, however, this parallel workflow in which preliminary simulation guides the experimental response characterization which then acts to improve the simulation prediction eliminates the need for as many iteration steps and then accelerates the overall product design cycle.
Conclusions
In conclusion, Azure Percept has been a welcome addition to an immersive design project that seeks to optimize product design through life cycle testing.
Future work is aiming to focus on performing more of the image analysis algorithms locally on the Percept device, or in the cloud. Furthermore, using more innovative ways for anomaly detection for the experimental testing control is also a focus of the project moving forward.
Overall, the students who have participated in a pilot (remote) offering of a graduate course where this design activity was given had given resoundingly positive feedback as to the interesting scope of the project, as well as the unique design workflows that are relevant in both industry and academia alike.
To Learn more about Azure Percept check out the following resources:
Azure Percept product page
Product detail pages
Pre-built AI models
Azure Percept - YouTube
Azure Percept DK and Azure Percept Audio now available in more regions! - Microsoft Tech Community
Posted at https://sl.advdat.com/3o8jxzh