The label type you choose sets both your model's ceiling and your annotation budget. Too simple and the model cannot learn the shapes it needs; too detailed and you pay for precision you never use. Here is how the main image annotation types compare.
The main types at a glance
| Type | What you draw | Precision | Relative effort | Good for |
|---|---|---|---|---|
| Classification | One tag per image | None (no location) | Lowest | Sorting, content moderation, quality pass/fail |
| Bounding box | A rectangle around each object | Rough | Low | Object detection, counting, retail, traffic |
| Polygon | Points tracing the object's outline | Good | Medium | Irregular shapes: buildings, fields, damage areas |
| Semantic segmentation | A class for every pixel | Very high | High | Drivable area, land cover, medical regions |
| Instance segmentation | A separate mask for each object | Very high | Highest | Robot picking, cell counting, overlapping objects |
| Keypoints | Named points (joints, landmarks) | Point-level | Medium | Pose estimation, faces, product measurements |
| Polylines | Open lines | Line-level | Low to medium | Lanes, roads, power lines, pipelines |
| 3D cuboids | A 3D box with heading | Spatial | Medium to high | Depth and orientation; see the LiDAR guide |
Bounding boxes: fast and often enough
Boxes are quick to draw and to review, and most detection models are built for them. Their weakness is that they include background: a box around a diagonal pole or a bent arm is mostly empty space. Use boxes when you need to know what and roughly where.
Polygons: the middle ground
Polygons follow the outline, so they capture shape and exclude most background. They suit objects with irregular edges, and they are the usual choice for aerial and satellite imagery, such as building footprints and field boundaries.
Segmentation: when every pixel matters
Semantic segmentation tells the model which class each pixel belongs to, but does not separate two touching cars. Instance segmentation separates them. Both cost more, though AI-assisted tools (promptable segmentation models that turn a click into a mask) have cut the effort a lot; annotators then refine the edges.
A simple decision guide
- Do you only need to know if something is in the image? Use classification.
- Do you need where, and rough location is enough? Use bounding boxes.
- Does the shape matter, or do boxes include too much background? Use polygons.
- Do you need exact area or boundaries? Use semantic segmentation; if objects touch or overlap and must be counted, use instance segmentation.
- Is the target a pose or a line? Use keypoints or polylines.
You can mix types in one project: boxes for most classes and masks only for the classes where shape matters. That alone can cut cost a lot; see how annotation is priced.
Test before you commit
Label a small sample two ways, train quick baselines and compare. If boxes get you close to your accuracy target, the cheaper option wins. A free Annova pilot is a good place to run that test, and our image and video annotation team supports every type above.
