Real-Time Big Data Monitoring

The GEODIS IT department lacked visibility into the actual usage of its Power BI and Graph API resources, leading to unjustified licensing costs and reactive incident management. To solve this, I developed and industrialized a centralized monitoring pipeline to process this massive data continuously. My hybrid architecture on Cloudera leverages Apache Kafka and NiFi for massive log capture, paired with Dataiku DSS for data cleaning and semantic enrichment. This system identified 20% of inactive licenses and automated error detection, cutting technical incident resolution time by half.