Background
Atlas Machine & Supply (“Atlas”) is a heavy-capacity industrial machinery manufacturing company specializing in areas such as compressed air and machining and repair. Their work has been delivered across virtually all industries that produce physical processes via machinery. Since 1907, Atlas has expanded its reach throughout the United States and positioned itself as a leading provider of manufacturing and repair needs, contributing to high job volumes across their eight facilities.
Challenge
In order to schedule and monitor the high volume of manufacturing jobs, Atlas utilizes scheduling software called Just Plan It. This software houses job and task data such as planned and actual start and end times, statuses, resources, and capacity that all add to an increased understanding of organizational productivity. While valuable data was collected in Just Plan It, retrieving that data was a manual and time-consuming process, making internal reporting a difficult task.
Without a direct pipeline, Atlas would have to continue to access this data via manual pulls, delaying access to critical organizational productivity and scheduling data. Atlas approached Keyrus as they looked for a solution to pull their data from Just Plan It on an automated schedule and land it in Amazon Redshift to facilitate internal reporting in Tableau.
Solution
Through conversations with the Atlas team, Keyrus learned about the availability of Just Plan It’s production scheduling API, and explored potential integration options. Keyrus recommended developing a pipeline that would effectively connect Tableau to the scheduling data via a multi-tool solution.
In order to facilitate automation, Keyrus utilized Amazon Sagemaker as an intuitive and cost-effective tool to run a Jupyter notebook on a daily schedule. Within the Jupyter notebooks were blocks of code utilizing Python’s boto3 library that enabled data ingestion, data structuring, and creation of tables and views in Redshift.
Lambda functions were first used to ingest data from the REST API, which was loaded into S3 buckets in parquet file format. AWS Glue Data Catalog and AWS Glue Crawler were then used to define the table schema of the parquet files. Once table definition is defined in AWS Glue Catalog, the notebook utilizes Redshift Spectrum to create external schema in Amazon Redshift to which tables and views are created that may be used as data sources in Tableau.
Results
Today Atlas no longer needs to spend hours per month manually pulling and organizing the high volume of manufacturing jobs data. Atlas managers can spot status, resource, and capacity issues, drill into root causes, and communicate results with the time that they used to spend just collecting the data. This is critical because continuously improving organizational productivity directly affects Atlas’ bottom line.
Keyrus & AWS Keyrus is a global consultancy specializing in building modern analytics platforms in the cloud. We are an Advanced Tier AWS Consulting partner with over 20 certifications, 3 accredited service delivery programs (EC2 for Windows Server, Service Catalog, and Redshift) and are part of 2 partner programs (Solution Provider and Public Sector Partner).