An automated video editing system uses user inputs and metadata combined with machine learning technology to gradually improve editing techniques as more footage is edited. The system is designed to work primarily with a network of automated video recording systems that use cooperative tracking methods. The system is also designed to improve tracking algorithms used in cooperative tracking and to enable systems to begin using image recognition based tracking when the results of machine learning are utilized.