You are leaving our main website to go to our chinese website hosted in China. For legal reasons there will not be any links pointing back to the main website.

Go to chinese website
Logo - Keyrus
Logo - Keyrus
  • Playbook
  • Services
  • Insights
  • Partners
  • Careers
  • About us
    Company purpose
    Innovation & Technologies
    Committed Keyrus
    Regulatory compliance
    Investors
    Management team
    Brands
    Locations

Success story

Modernizing a data platform at a global trade organization

by Norman Chea

Background

A large global trade organization (GTO) has a mission to promote the sciences in order to improve the quality of life for all. The organization focuses on publications and membership, with a membership exceeding 150,000 in more than 140 countries. They publish over 70 peer-reviewed journals that are considered cutting edge in research within their domain. Additionally, they organize national and regional conferences to promote the sciences.

Challenge

Keyrus was brought in to work with the GTO to address very specific meta-data management and data cataloging concerns that their business stakeholders had identified. The existing data platform was a 15+-year-old Oracle system that had supported the business well over the years, but had complex logic that was not documented. 

Whenever there were questions on how KPIs were being added into the system, it was a very time-consuming process to figure out what the existing data transformations were doing. To understand what was done, it would require IT to reverse engineer the layers upon layers of code and then document these findings back to the business. 

This ultimately created bottlenecks for their lean development team and instilled a lack of confidence in some of the metrics, due to the black box logic. 

Approach and Key Success Factors

Discovery and Learning the Business

Keyrus executed a data discovery to assess their entire data platform and provide recommendations on how to resolve their meta-data management needs. This was a 5-week discovery where our team conducted interviews across their key department stakeholders to understand existing data processes, how they use the data, and what capabilities they would like in the future. 

We determined that meta-data management was only part of the challenge, and that a recommendation for a data cataloging tool wouldn’t solve all the core issues. While the 15+-year-old system did its job at meeting existing business requirements, there was a need to modernize the platform to accommodate future requirements. 

Some of the core future requirements that were discovered were:

  • A data platform that can scale for growing data volumes and connections without spending significant overhead on IT time

  • Modernize their BI consumption layer and provide self-service capabilities for business users to generate their own insights

  • Remove the need to provide manual reports and dashboards to external stakeholders

  • Provide a more transparent data lineage capability to identify issues

At the end of the discovery, we presented our findings to the main stakeholders at the GTO and proposed a data platform modernization to address the core challenges that expanded outside of the data lineage challenges.

We provided the GTO leadership team with recommendations on future technology stack options, based on the requirements we gathered from business and IT. To supplement the tool comparison, we also conducted a two-week proof-of-concept to demonstrate the capabilities of handling the GTO’s current and future state requirements. We chose Snowflake, Tableau, and Talend and agreed on a plan to execute the modernization effort.

Data Platform Modernization

At the implementation stage, our team collaborated with their stakeholders to design and build the future state data platform. First, we reverse engineered the 15-year-old system and looked at a large library of their Java classes and Oracle stored procedures to translate complex and layered logic into readable Talend data pipelines. Creating these readable flows was easier to follow and maintain than existing Java logic, and since Talend leverages a graphical user interface, it enables a larger pool of their developers to generate their own data pipelines, as opposed to being reserved for Java developers. 

Lastly, Talend’s ETL tool offers great native connectivity to all of the connections that the GTO would like to leverage in the future, which reduces the time developers would have to spend building a custom connection in the existing solution.

To address their scalability requirement, we leveraged a fully cloud solution, leveraging Amazon Web Services (AWS) to host the Talend ETL engine and Snowflake for their data warehouse. These tools provided a highly elastic solution that would enable the team to scale their resources up and down accordingly. 

Leveraging a cloud-based and managed solution allows the GTO to focus on generating business value as opposed to upkeep and patching on-premise infrastructure. On top of the natural scalability of the tools, Keyrus also leveraged Cloud Formation scripts on the AWS side that helps automate standing up the infrastructure.

To create a modern BI consumption layer, we leveraged Tableau as the enterprise reporting tool. Our first effort on the consumption layer was to factor the redundant reports and dashboards that existed within their legacy BusinessObjects stack. 

Our team worked closely with key stakeholders to consolidate 30 dashboards and reports into six Tableau dashboards. This instantly created a better user experience, so that users can go to one dashboard as opposed to several to get the data sets they are looking for. 

This also reduces the maintenance burden of IT by consolidating these objects. We also collaborated with the GTO’s infrastructure staff to install and deploy a Tableau external server, which allows end users who are outside of their organization’s network to access dashboards and reports. With this external server, their business intelligence team can now deploy an automated dashboard that their external users can access at any time, as opposed to building out manual reports every month.

To tie this all together, we deployed a Talend Data Catalog solution to address the data lineage requirements. This tool provides great native integration with our entire recommended stack (Talend ETL, Snowflake, and Tableau). 

Keyrus configured the data catalog to automatically harvest and read the metadata across these different technologies, which automatically create end-to-end data lineage. This allows business users to see the entire journey of all their data assets - how a data element goes from source to data warehouse to dashboard - and how it gets transformed in the entire data process. This provides great visibility that the GTO stakeholders never had before, allowing them to better validate and trust their data.

 What Made Us Successful

  • Our team leveraged best-in-class technologies and layered them together to provide GTO with a custom solution that met a wide range of business needs. 

  • Through proven technical expertise implementing data platform modernizations, we leveraged accelerators (i.e., Cloud Formation scripts) to reduce development time.

  • Managing different software vendors with different license and pricing structures can be complex. Keyrus operated as a reseller thanks to our relationship with different software partners. This created a one-stop-shop for our customer to get all of their software licenses they needed across different vendors.  

Key Deliverables 

  • Comprehensive assessment of existing infrastructure (ERD diagrams, workflows, interview notes)

  • Vendor technology comparisons for their entire data infrastructure stack (i.e., data visualization and analytics, data warehousing, ETL, and infrastructure), including executing a proof-of-concept

  • Detailed design of the new solution inclusive of new server/network infrastructure, data warehouse structure and models, error handling, and reporting principles

  • Reverse engineered a 15+-year-old system built on legacy Oracle and Java code into a modernized data platform

  • Stood up an external Tableau server to enable the new data platform to be accessed by their end users outside the company’s network

  • Conducted training on the new platform to a wide range of audiences, including a larger Tableau training

whatsapptwitter
linkedinfacebookworkplace
newsletter.svg

Never miss an insight

Stay updated on the latest articles, events, and more

Your email address is only used to send you the Keyrus newsletter and for commercial prospecting purposes. You can use the link in our emails to opt-out at any time. Learn more about the management of your data and your rights.

Continue reading

Press release

Keyrus named amongst Top B2B Companies on Clutch

December 12, 2022

The Keyrus team is excited to announce that we’ve been named one of the top 1000 companies on Clutch’s platform in 2022! This is the second year that Keyrus has been recognized by Clutch as a top company and B2B leader. 

Webinar

PDF Parsing with Alteryx Intelligence Suite

December 1, 2022

In 20 minutes, we’ll teach you how to use Alteryx Intelligence Suite to eliminate common problems and inefficiencies in accessing data from .pdf files. In the past, you’d need to run custom Python and complex parsing logic to get any usable data from a pdf. Now, you can parse PDFs with out-of-the-box features in Alteryx Intelligence Suite.

Webinar

Modern Cloud Analytics in Action: Keyrus and Red Ventures

November 11, 2022

The cloud offers new opportunities to save you time and money, allowing you to shift focus from maintaining growing servers and upgrading infrastructure to making your data work for you and the success of your business. Watch the webinar and Q&A to learn how AWS, Tableau, and Keyrus worked together to help Red Ventures migrate to a powerful cloud BI tool that created new pathways for success and a modern data culture.

Event

Pharma/Biotech GTN Summit 2022

October 27, 2022

Keyrus & Anaplan Sponsor Life Science Gross-to-Net (GTN) Summit

Press release

Keyrus Achieves AWS Data and Analytics Competency Status

October 6, 2022

Keyrus achieved Amazon Web Services (AWS) Data and Analytics Competency. To receive the designation, AWS Partners must possess deep AWS expertise and deliver solutions seamlessly on AWS.

Webinar

Live Webinar: Lessons on workforce capacity planning and optimization from Optum (UnitedHealthcare)

October 19, 2022

Wednesday, November 9th, 2022 @ 12:00PM Central Time (US and Canada)

Webinar

Tableau Embedded Analytics: Optimizing insights from Salesforce data

September 20, 2022

Want to optimize your visual analytics in Salesforce? You need the right tools. Tableau Embedded Analytics can be used to help you build and visualize reports in Salesforce.

Success story

How C&S Wholesale Grocers maximized ROI with an analytics center of excellence

September 7, 2022

C&S Wholesale Grocers worked with Keyrus and Alteryx to implement an analytics center of excellence to help them efficiently and effectively achieve business objectives, maximize return on investment (ROI), and standardize best practices.

Success story

Implementing a cloud security automation tool at a global consulting firm

September 2, 2022

Keyrus partnered with a consulting firm to build an in-house cloud security solution that would automate their verification processes and keep their information safe.

Success story

Leveraging Salesforce to improve operations at Pajama Program

July 25, 2022

Keyrus partnered with Pajama Program, a nonprofit organization, to review their Salesforce architecture and improve overall operations.

Logo - Keyrus
New York City

252 West 37th st., Suite 1400 New York, NY 10018

Phone:+1 646 664 4872

LinkedInInstagram
PlaybookServicesInsightsPartnersCareersAbout us
Company purposeInnovation & TechnologiesCommitted KeyrusRegulatory complianceInvestorsManagement teamBrandsLocations
Legal notice & Terms of use
Privacy policy
Data protection