English is currently available.
EdgeAI DISCUSSION
For a vibration or audio classifier, is it valid to create many overlapping windows and then randomly split those windows into training and test sets?
Adjacent windows may contain much of the same signal. How should the data be divided so the reported result better predicts performance on a new recording session, device or operating condition?
Split by the independent unit you want to generalize to before creating overlapping windows. That might be a recording session, machine, person or physical device. If two windows share most of their raw samples, putting one in training and the other in testing gives the model information about the test recording. A high score can then overstate performance on truly new data.
Keep preprocessing decisions inside the training workflow too. Fit normalization parameters and select features using training data, then apply those fixed choices to validation and test data. If the operating condition changes over time, inspect chronological or grouped evaluation as appropriate. Record how the split was made so future model comparisons use the same definition of unseen data.
Test the deployed pipeline on complete new recordings, including transitions, silence or idle periods and conditions outside the known classes. Window-level accuracy can hide repeated false triggers over a long run. Report event-level behavior and the smoothing or voting logic used by the application. Those measurements are closer to what a device user experiences than an isolated-window score.
Back to EdgeAI community
