Logo - Keyrus
  • Playbook
  • Services
    Data advisory & consulting
    Data & analytics solutions
    Artificial Intelligence (AI)
    Enterprise Performance Management (EPM)
    Digital & multi-experience
  • Insights
  • Partners
  • Careers
  • About us
    What sets us apart
    Company purpose
    Innovation & Technologies
    Committed Keyrus
    Regulatory compliance
    Investors
    Management team
    Brands
    Locations
  • Contact UsJoin us

From Manual to Magic: File Ingestion Pipeline with Anaplan

Developed a sustainable solution to ingest and integrate site excel files into Anaplan.

Triggered approach to file ingestion.

Background

This is a multinational mining company based in Canada as part of an Anaplan implementation required some work to support their ingestion and transformation process by leveraging their new Databricks environment and loading Excel files (stored in SharePoint for their multiple mining sites), transformed into Anaplan for consumption.

Challenge

The challenge here is Local Excel files that only have a manual way of uploading information to Anaplan, causing a lack of insight into site data for the analytics team.

Approach

Keyrus implemented a synchronized S3 bucket with SharePoint to directly access the files from the S3 bucket in real time. The team developed several Databricks notebooks to load the files, triggered whenever they are updated, into a medallion architecture in the unity catalog. The Bronze layer is used to store the raw data and the Silver to load the transformed data. The transformations were different for each category and were made using Spark. Once the data was transformed, we had to upload it in CSV files to S3 to load them into Anaplan. The loading process was made with Anaplan “CloudWorks” by choosing the right file in the s3 and the right action (process). This Silver layer data was also made available for the analytics team providing site insights they never had to use across other business use cases like ESG Reporting.

Key results

01
A little over 200 hours to build out the secure pipelines and transformations validating ingestion focused on Maintenance, Engineering, HR, and FP&A detail.
02
Initially scheduled with a daily refresh schedule but upgraded the process to trigger on file update in SharePoint to shorten the ingestion timeline into Anaplan.

Benefits

Robust pipeline delivering required information with actions to process it with Anaplan. Access to data that was not available or being collected from sites previously.

Technology partners

AWS

Amazon Web Services (AWS) is a global leader in cloud-computing services that offers over 200 fully featured services from data centers worldwide. AWS provides services to organizations of all sizes to help lower costs, become more agile, and innovate faster.

Advanced Tier Partner

with a Data & Analytics Competency

30+

General Certifications

15

Speciality and Professional Certifications

Share this key play

whatsapptwitter
linkedinfacebookworkplace

More key plays

Enhancing Operational Process Performance with a Modern Cloud Platform
Smarter, Faster, and Scalable: How Keyrus Leveraged GenAI and UNSPSC for Product Classification
Faster Power BI Reports and Better Use of Premium Capacity: How We Helped Pernod Ricard Do It
How We Enabled Massport to Gather Data about their Infrastructure Conditions to Achieve Agency's Mission and Goals
Transforming ESG Reporting: Automating Metrics Collection with Databricks Integration
Boost sales with targeted marketing campaigns and multi-channel communication
Data, Technology & Digitalization for One of the Biggest Agro Companies in Brazil
Gain new insights to optimize logistic processes and reduce costs by leveraging advanced analytics
Using Alteryx to automate processes at an international bank
Enabling data-driven decisions at a large alcoholic beverage company
Logo - Keyrus
Montreal

1396 rue Sainte Catherine Ouest #205 Montréal QC H3G 1P9 Canada