Introduction
Industrial safety has always been a critical pillar in sectors like oil, gas, and manufacturing. Ensuring compliance with safety equipment like helmets, vests, and shoes can significantly reduce workplace hazards. Traditional manual inspections, however, often fail to detect violations in real time, leading to safety risks, inefficiencies, and non-compliance issues.
This blog highlights a cutting-edge AI-powered Machine Vision (AI-MV) solution developed using Azure Databricks and other Azure services in Acuprism(Enterprise Data & AI Platform). It automates the real-time monitoring of safety violations using live video feeds from CCTVs, drones, and robotic cameras — driving compliance, minimizing risks, and optimizing safety operations.
What is AI-based Safety Monitoring?
AI-based safety monitoring is a proactive system that uses real-time computer vision and machine learning models to detect compliance violations, such as missing personal protective equipment (PPE). Video streams from factory floors are analyzed live, and any detected safety issues (e.g., no helmet or vest) are flagged automatically.
This enables organizations to monitor safety 24/7 without relying on human inspectors — leading to higher compliance, improved response times, and reduced incident rates.
Benefits of AI-MV in Industrial Safety
- Real-Time Detection: Identifies violations instantly, minimizing risk.
- Improved Compliance: Ensures all personnel wear appropriate safety gear.
- Reduced Manual Monitoring: Automates the process, freeing human resources.
- Cost Efficiency: Lowers incident-related costs through proactive detection.
- Centralized Insights: Uses dashboards and reports for instant decision-making.
A Case Study: Vision System Implementation in Oil & Gas
A major oil and gas company aimed to automate the monitoring of PPE compliance across their production lines. Traditional manual inspection processes were not scalable and often delayed critical responses to safety breaches.
The solution — AcuPrism AI-MV System — was implemented using Microsoft Azure services, Databricks, and Power BI, bringing real-time intelligence and automation into the company’s safety workflow.
Architecture: AcuPrism on Databricks
The solution follows a modular and scalable architecture:
- Azure Blob Storage for storing image/video data securely.
- Databricks for data preprocessing, model training, and inference.
- MLflow for model tracking and versioning.
- Azure Function App for processing and integration with Kafka.
- Kafka for ingesting real-time video streams.
- Power BI for visualization and alerting.
End-to-End Workflow
- Data Collection: Video frames are streamed from CCTV, drones, and robots.
- Storage & Access: Frames are ingested and stored in Blob containers.
- Annotation: Images are annotated in Roboflow to train detection models.
- Training: YOLOv8 is trained using annotated datasets in Databricks.
- Model Deployment: The trained model is deployed to a Databricks Serving Endpoint.
- Real-Time Inference: Kafka streams video frames to Databricks for live detection.
- Insights: Results are visualized in Power BI dashboards.
- Actions: Power Automate triggers alerts and logs entries into SharePoint.
Model Training & Deployment
- Data Collection: Video frames are streamed from CCTV, drones, and robots.
- Storage & Access: Frames are ingested and stored in Blob containers.
- Annotation: Images are annotated in Roboflow to train detection models.
- Training: YOLOv8 is trained using annotated datasets in Databricks.
- Model Deployment: The trained model is deployed to a Databricks Serving Endpoint.
- Real-Time Inference: Kafka streams video frames to Databricks for live detection.
- Insights: Results are visualized in Power BI dashboards.
- Actions: Power Automate triggers alerts and logs entries into SharePoint.
Power Apps: User Interface Triggering the Flow
This is the starting page of the Monitoring HSSE system, which consists of three key sections where cameras modifications and surveillance starting can be made as required.
- In first section admin can login and authenticate with a username and password.
- Production Line Selection: Admin selects the production line (e.g., Line 1, Line 2, etc.).
- Start Surveillance: Upon selection, Power Apps sends Kafka credentials and trigger details to Azure Function App.
- This interaction ensures that only authorized personnel can initiate and configure monitoring operations, enabling secure and controlled safety enforcement.
- Second section it is for viewing and modifying the Camera Configuration details. it also has the option to delete any existing details if needed.
- Third section is for viewing the dashboard, after clicking on the click here button it will redirect to a power BI dashboard.
Real-Time Processing Using Kafka & Function App
- Kafka Producer reads frames from RTSP streams.
- Frames are sent to Confluent Cloud Kafka topics.
- Azure Function App receives frames, sends them to the model for prediction.
- Detected results are uploaded back to Blob Storage.
- Alerts or notification will be sent to mail and teams to the respective authorities for taking action.
Visualization in Power BI
- Shows violation counts in real time.
- Tracks compliance trends across production lines.
- Sends real-time alerts to Microsoft Teams using Power Automate.
Business Impact and Results
Below is the test image for detecting safety violation detection.
- Increased Compliance: 80% reduction in undetected violations.
- Faster Response Times: Immediate alerts enabled quicker resolution.
- Reduced Manual Effort: 60% cut in monitoring hours.
- Data-Driven Decisions: Better planning using historical safety trends.
Future Scope
- Edge Deployment: Extend model to run on edge devices for ultra-low latency.
- Behavioral Safety Analytics: Detect risky human behavior beyond PPE violations.
- Integration with ERP Systems: Enable automated incident management workflows.
Conclusion
The AI-powered Machine Vision system marks a significant leap in industrial safety. By combining the power of Databricks, Azure ML, Kafka, and Power BI, organizations can now move from reactive safety practices to proactive, real-time safety enforcement. This not only improves compliance but also ensures a safer, smarter, and more efficient workplace.
