AI-Powered Medical Surgery Recording App

• Computational Limitations: The inability to utilize servers for computational tasks necessitated the development of a solution capable of handling near real-time processing tasks locally on the device.
• Sound Data Collection: Detecting sounds in video footage to recognize events.
• Video Data Collection: Segmenting the footage, and identifying objects.

Two neural network architectures: for sound recognition and for object segmentation were selected for their compatibility with the CoreML framework and proven quality. The development process for both networks involved several critical steps:

• Data Collection: A large and representative sample of data from as many as possible various environments was prioritized to enhance the relevance and accuracy of neural network training.
• Preprocessing and Augmentation: Refining and enhancing the collected data to prepare it for neural network training. This step is crucial for improving model accuracy and robustness.
• Model Training, Validation, and Evaluation: Rigorous training of the Wavenet and Unet models followed by validation and evaluation to ensure they met the desired quality criteria.
• Model Conversion: Both models were converted into the .mlmodel format, enabling them to run efficiently on iOS devices.