How should an int8 edge model be compared with its float32 version?

Use the same held-out test set for both models and inspect the confusion matrix, per-class recall and precision, not just total accuracy. An uncommon class can degrade substantially while the aggregate changes little. Include representative operating conditions and inputs near class boundaries. Keep the test data separate from the data used to tune the model or choose its quantization settings.

Measure latency, RAM, flash use and energy on the actual target with the complete preprocessing path. A smaller weight file does not guarantee proportionally smaller runtime memory or faster inference for every operator and backend. Use identical input samples and report the model version, compiler settings and target clock so another person can reproduce the comparison.

Check decision thresholds after conversion as well. If application logic acts only above a confidence threshold, a modest shift in output scores can change its behavior even when the winning class stays the same. Test the final decision pipeline, including an unknown or reject outcome where appropriate. Choose the model against the application's acceptable error and resource budgets.