Jun 24

Performance Measurement Metrics for Target (Aeroplanes, People) Tracking

Performance Measurement Metrics for Target (Aeroplanes, People) Tracking

By Sayed Ahmed

Let’s see, you want to track the aeroplanes in the sky using some radar and tracking algorithms. You have written simulation software for that. How will you measure that your software and algorithm is doing a great job. Even in real life, how will you know that your aero plane tracking is working well?

You can define some metrics for that i.e. to measure the performance.

T: Tracks : your decision/finding of the aero plane trajectory

L: Truths : The actual trajectory of the aero planes

M: Measures: your radar measurements

What is the general performance metric? What does it mean?

  • p = g (M, L, T )

What are performance categories?

  • Sensor Related: Independent of tracking result (Cramer Rao Lower Bound, Sensor Characteristics)
  • Tracker Related : Tracking Result, Tracker Type, application of interest

What are the subdivisions of Tracker Dependent Metrics?

  • Algorithm free: Based on Available Truth and Unavailable Truth (tracking and truth)
  • Algorithm dependent : application dependent : For MHT, Dynamic Programming

Algorithm Free

  • Cardinality Measures, based on some numbers
    • Count of Valid Tracks, false tracks, missed tracks, spurious tracks, completeness, broken tracks, swaps in tracks
  • Time Measures: Performance based on Time Measures
    • Rate of false alarm, redundant track ratio, probability of detection, fragmentation rate, track latency
  • Available Truths
    • Truth to Track Association -> association matrix at each scan and monte carlo run
    • Distance metric -> between truths and tracks
  • Unavailable Truths:
    • Consistency of estimations -> To innovation of estimation
  • Track Cardinality Measures
    • Track result and Truth Data
    • Valid Tracks: A target is assigned to one track, the target is not assigned to any other
    • Missed Tracks: If a target is not assigned to any tracks
    • False Tracks: No target is assigned to that track, only false alarms for example
    • Spurious Tracks: A track is assigned to multiple targets
    • Completeness: Number of valid tracks per truth
    • Swaps in Tracks: Track Crossings by targets. Count of track segments for each target over the total time
    • Broken Tracks: For each timestamp, there is no track assigned to the truth. Count B (l, m) : l_th truth, m_th run. Count of broken tracks
  • Time Measures
    • Track Latency
      • Confirmation latency: does it mean target start to track confirmation
      • Termination Latency : does it mean target disappear to termination time
        • Timestamp i.e. time required
    • Rate of False Alarm:
      • Average number of false tracks per time
    • Track Probability of Detection
      • ( duration l th target is assigned to a valid track) / (l th track last scan time – l th track first scan time)
  • Accuracy Measures
    • RMSE:
    • kth scan, l th target, M monte carlo run : Mth monte carlo
  • Average Euclidean Error:
    • RMSE without Square Root, no square in the error
  • Geometric Average Error:
    • Average Euclidean Error, just sum of log
  • Harmonic: 1/(sum(1/error)/Cont_monte_carlo_run )

Sensor Metrics

  • Probability of detection: within scenario, within FOV
  • Coverage: Average, minimum, maximum
  • Detection Gap: Average, Minimum, Maximum

Estimator Consistency

  • Normalized Estimation Error Squared

Selected Metrics:

  • Number of false tracks,
  • track probability of detection,
  • fragmentation rate,
  • track latency (confirmation, deletion),
  • swap count,
  • RMSE

Score:

Un-normalized score, Normalized Score, Best = 1.00 , worst = 0.0

References:

A. A. Gorji ; R. Tharmarasa ; T. Kirubarajan: Performance measures for multiple target tracking problems

*** . *** *** . *** . *** . ***

Training Courses: http://Training.SitesTree.com
Blog: http://Bangla.SaLearningSchool.com, http://SitesTree.com
8112223 Canada Inc/Justetc: http://JustEtc.net
Youtube: https://www.youtube.com/channel/UCHrtq0pV8mEuTLGd2-Lp9ig/playlists
Medium: https://medium.com/@SayedAhmedCanada


Sayed Ahmed

BSc. Eng. in Comp. Sc. & Eng. (BUET)
MSc. in Comp. Sc. (U of Manitoba, Canada)
MSc. in Data Science and Analytics (Ryerson University, Canada)
Linkedin: https://ca.linkedin.com/in/sayedjustetc

Facebook Groups/Forums to discuss (Q & A):
https://www.facebook.com/salearningschool

https://www.facebook.com/banglasalearningschool
https://www.facebook.com/justetcsocial

Get access to courses on Big Data, Data Science, AI, Cloud, Linux, System Admin, Web Development and Misc. related. Also, create your own course to sell to others. http://sitestree.com/training/

Join, Discuss, and Thrive:
Finance: https://www.facebook.com/groups/1792339521056034

Bangladesh and Canada: https://www.facebook.com/groups/398414183875009

Bangladesh: IT & Eng: Canada: https://www.facebook.com/groups/480297675634630

BUETians in Ontario: https://www.facebook.com/groups/371982239854027

Worldwide IT: https://www.facebook.com/groups/28362061867

JustEtc: https://www.facebook.com/JustETC-Technologies-Inc-289864738261