How to Train Pixy Camera with Computer

Hello friends! Our Pixy camera is connected and PixyMon is working, so we can now teach the camera which colour we want it to detect. In this tutorial, we will select a blue region on a globe, store it as a signature and refine the detection when other parts of the scene also match.

We are using the original Pixy CMUcam5 and its matching PixyMon software. The screenshots belong to that setup. If you have Pixy2, use its corresponding software and instructions rather than assuming that every menu and parameter is identical.

The most useful idea to understand before starting is that we are teaching a colour signature. We are not teaching the camera the meaning of a globe or asking it to recognize every globe regardless of colour. Once that distinction is clear, the results and their limitations become much easier to explain.

What Does Training a Pixy Camera Mean?

In this workflow, training provides representative image pixels from which the camera builds a colour signature. Later, matching regions can be reported as detected blocks. The selected pixels therefore determine what the camera is trying to find.

A blue area of our globe is a convenient example. Another sufficiently similar blue object can also match, even if it has a completely different shape. Conversely, a different part of the same globe may not match if its colour differs from the selected region.

Think of the signature as a practical selection rule rather than a complete object description. This approach works especially well when you can choose a distinct marker and control the background around it.

The manufacturer's original-Pixy teaching guide describes the computer-based selection process and the use of up to seven colour signatures. The signature number is a label; signature 1 is not restricted to a particular target colour.

Prepare the Camera and Scene

Connect the camera to the computer with the appropriate USB data cable and launch the matching PixyMon application. If the camera is not detected, complete the PixyMon installation and connection checks before changing detection parameters.

Place the target where you expect to use it. Begin with steady lighting and a background that is visibly different from the target colour. Keep the camera stationary while choosing the training region.

The original example shows a globe in the live view. We will use its blue surface as our target colour. Avoid selecting a region containing a mixture of blue, land markings, a bright reflection and the background.

Check that the image updates as the scene changes. A connection that produces live video is a separate milestone from successful detection. If the image is frozen because a selection operation is active, finish or cancel that operation before diagnosing a USB fault.

Select Signature 1

Open the Action menu and choose Set signature 1. The original screenshot shows the corresponding menu selection.

Use signature 1 for the first target so the result is easy to identify. Record what that number means in your project. For example, write “signature 1: blue globe surface” rather than only recording “blue,” which may become ambiguous when several blue items are present.

Teaching an existing signature changes its meaning. If another application already depends on that number, record the previous setup before replacing it. A controller that reacts to signature 1 cannot know that you have changed the target associated with the label.

Choose a Representative Region

Click and drag over a useful patch of the target colour. The selection view may pause while you choose the pixels. The original example selects part of the blue globe rather than the whole frame.

Choose a region that contains the colour you want the camera to learn, with enough area to avoid relying on one accidental pixel. Keep obvious background pixels outside the selection.

A larger selection is not automatically better. If it includes multiple unrelated colours, it may describe a broader distribution than intended. A tiny selection from a shiny highlight can be equally misleading because the highlight may not represent the normal surface.

For a curved object, begin with a patch that is visible under the normal working illumination. Then evaluate the result as the object rotates. If rotation reveals a substantially different surface colour, one signature may not represent the entire appearance you hoped to detect.

After selection, inspect the processed view and the signature labels. The manufacturer calls this processed display cooked video. Its purpose here is to let us see which regions satisfy the learned rule.

Interpret the First Detection

The original result places a signature-1 label around the selected blue region. It also detects some smaller matching regions elsewhere in the scene.

This is an important observation. The camera has not necessarily malfunctioned; it may be correctly applying a colour rule that is too broad for our intended task. We need to distinguish the colour-matching result from the application requirement to select only the globe.

Inspect every reported region, not only the largest box. A background match that seems harmless on screen can become a false action when a controller later reacts to any detection of that signature.

A box is a convenient summary of the selected region. It is not a statement that every enclosed pixel belongs to a uniquely recognized object. Keep that distinction in mind when using block dimensions for later decisions.

Open the Signature Tuning Controls

Open the configuration dialog, using File and Configure or the corresponding gear control in the original application. Find the signature-tuning parameters for signature 1.

The original screenshot shows a signature range value of 2.02400. That value belongs to the illustrated scene and setup. It is not a universal calibration constant for blue objects.

The range setting changes how inclusive the signature is. Reducing it can exclude unwanted colour variations, while increasing it can admit target pixels that were previously missed. The right direction depends on the failure you are trying to correct.

Change the value in small steps while observing both the target and unwanted regions. Keep a note of the starting value so you can return to it if the adjustment makes the result worse.

Reduce Unwanted Matches Without Losing the Target

In the original experiment, reducing the signature range removed the small background detections while retaining the globe's blue region.

That is a useful improvement for this scene. It does not prove that the same setting will work when the globe moves into shadow or a similar blue object appears beside it.

After each adjustment, move the target through the positions that matter to your project. Then remove it. A narrower rule may look clean in one position but miss the target elsewhere.

Use Apply or OK to retain the configuration changes. Cancelling the dialog can discard the adjustment, so confirm the result after closing the settings window.

Separate Colour Range from Object Size

A smaller unwanted box may tempt you to reduce the signature range until it disappears. That can work when its colour differs enough from the intended target, but it is not the same operation as filtering by size.

If a distant intended target becomes small, a minimum-size rule may reject it too. If an unwanted object has the same colour and similar size, neither colour nor size alone can distinguish it reliably.

For a fixed scene, location can provide another useful condition. You might accept a matching block only when its centre lies within the part of the image occupied by a work tray. Record this as an application rule rather than pretending the camera has learned object identity.

Combine only conditions that have a physical reason. An arbitrary position restriction can fail when someone moves the camera, while an arbitrary width threshold can fail when the target distance changes.

Improve Lighting Before Over-Tuning

The manufacturer's detection-improvement guide discusses signature range, exposure and minimum brightness. It also describes overexposure highlighting, which helps identify image areas where strong illumination has removed useful colour information.

A shiny surface can produce a bright reflection that differs from its normal colour. Moving the light or camera may restore a useful view more effectively than widening the signature until large portions of the background are accepted.

Minimum brightness is another distinct condition. Raising it can reject dark unwanted regions, but it can also remove a target in shadow. Lowering it may recover that target while admitting dark background noise.

Change one parameter at a time. If you adjust exposure, range and minimum brightness together, you cannot tell which change caused the improvement or which introduced a new failure.

Lighting colour matters as well as brightness. A target viewed under one lamp may shift in appearance under another. When the installation lighting changes substantially, reevaluate the signature and white-balance setup instead of assuming the original training remains representative.

Build a Small Evaluation Set

Choose several repeatable scenes before declaring the signature finished. Include the target centred, near each edge, nearer and farther away, partly rotated and absent. Add at least one unrelated object with a similar colour.

For each scene, write down what should count as success. In a presence detector, one appropriate target box may be enough. In a positioning application, the box also needs a stable centre and useful dimensions.

Keep some examples separate from the scenes used to tune the settings. Otherwise, repeated adjustments may simply fit the few scenes you keep showing the camera without improving performance in new situations.

Calculate Useful Detection Measures

Suppose an evaluation contains 20 intended target presentations. The system detects the correct target in 18 and misses it in two. Recall for those presentations is 18/(18 + 2) = 0.90, or 90 percent.

If the same evaluation also contains three unwanted detections, precision is 18/(18 + 3), approximately 85.7 percent. This is an illustrative calculation, not a measured performance claim for the screenshots.

Precision and recall describe different errors. A narrower signature can reduce false detections while missing more targets. A wider signature can recover targets while accepting more background regions. Choose the tradeoff according to the application's consequences.

Define the counting method before collecting results. A persistent false box visible in 100 consecutive frames can be counted as 100 frame-level errors or one false event, but those measures answer different questions. Do not compare them as though they were the same statistic.

Check Position Stability

For a stationary target, record several reported centres and inspect their variation. If the box repeatedly grows and shrinks around a reflection, its centre can move even though the physical object has not moved.

For example, centres of 150, 152 and 151 pixels have a mean of 151 pixels. Centres of 130, 170 and 153 may have a similar average but a much larger spread. A control system needs to consider that variation, not only the mean.

Averaging can smooth a signal but adds delay and does not correct a systematically wrong target selection. First improve the detected region and the rule for choosing the intended block. Then decide whether temporal filtering is useful.

Keep image coordinates separate from physical coordinates. A centre at x = 150 identifies a location in the image, not a distance of 150 millimetres from the camera. Physical measurement requires an appropriate calibration and geometry model.

Add More Signatures Deliberately

Once signature 1 behaves acceptably, choose another distinct target and teach a different signature number. Maintain a small table mapping each number to its intended target or marker.

Test the signatures together. Two independently successful training exercises may interact when both targets appear in the same frame, particularly if their colours overlap under the actual lighting.

Do not label every possible appearance with a new signature simply to hide poor scene design. A clearer marker, a neutral background or better illumination may produce a simpler and more reliable project.

After changing a signature, repeat the relevant absence and distractor checks. A new setting can create a match in a region that previously caused no problem.

Prepare for Arduino Integration

Before connecting an actuator, confirm that the controller receives the same signature and block behaviour you inspected in PixyMon. Start with a serial printout of detections and move the target by hand.

Define an explicit response for no detection and multiple detections. A controller should not continue acting on an old coordinate indefinitely after the target leaves the scene.

A useful first application is an indicator that changes state only after several acceptable observations. For example, require three consecutive matching observations before declaring the target present, and separately define how absence clears that state.

This introduces a persistence rule, not additional visual recognition. At an illustrative 20-millisecond observation interval, three consecutive observations span about 40 milliseconds from the first to the third, with additional acquisition and application delays possible. Measure the complete system timing before using the rule in a moving mechanism.

Practical Review of the Globe Example

The original sequence demonstrates three useful steps: selecting a representative blue region, observing extra matches, and narrowing the signature to improve that particular scene. Preserve that reasoning when you work with a different target.

Do not copy the numerical range without reproducing the scene and checking the result. The selected pixels, lighting, camera settings and background all influence what that number means in practice.

Document the final conditions with a screenshot and a short parameter record. Include an example with the target absent. That second image often reveals more about practical reliability than another successful detection screenshot.

Troubleshooting

Frequently Asked Questions

Does selecting the globe teach its shape?

Not in this colour-signature workflow. The selected blue pixels define a colour class, and other matching regions can also be detected.

Should I always reduce the signature range?

No. Reduce it when unwanted colour variations are included. Increase it cautiously when useful target pixels are excluded, and reevaluate both error types.

Why can a small background object still matter?

Your controller may react to any matching block unless you add a justified selection rule. A small false match can therefore produce a real application error.

Do I need to teach the signature after every normal restart?

The original Pixy stores taught signatures in flash. Nevertheless, confirm the expected configuration after updates or deliberate resets.

What is the best sign that training is complete?

The target is detected under the intended conditions, unwanted scenes are handled acceptably, and the results can be repeated using recorded settings. One clean screenshot alone is not enough.