This is a preview of your story
analytics_default
analytics_default

Strategic Blueprint: Finding an ETL solution for FinTech Company


⏱ Avg. Reading Time: 10 min

expand_more

This strategic blueprint suggests actionable recommendations for resolving FTC's ETL solution challenges. It aims to bridge the gap between on-premise and cloud infrastructure, handle regularly changing Excel files with strict SLAs, and integrate with a Linux-based scheduler. Our strategies aim to provide a cost-effective solution that meets all client requirements.

Problem Statement

FTC needs a new ETL solution that replaces our outdated tool, aligns with our technology roadmap by integrating on-premise and cloud infrastructures, improves technical capabilities, processing speed, meets stringent SLAs, and enhances overall agility.

analytics_default

Key Objectives:

  • check_circle_outline

    Choose an ETL solution that bridges the gap between on-premise and cloud infrastructure.

  • check_circle_outline

    Implement an ETL tool that enables component reuse across multiple clients, leading to a streamlined code base and reduced maintenance efforts.

  • check_circle_outline

    Utilize the new ETL tool to meet client SLAs, enhance revenue generation by creating client-driven processes, and expand the available dataset for clients.

Key Findings & Analysis

Legacy ETL tool is no longer supported

The lack of vendor support for the ETL tool currently utilized by FTC for this product jeopardizes its revenue potential.

Numerous ETL solutions currently licensed

Currently utilize 5 ETL solutions across 8 products that lack integration between on-premise and cloud infrastructure.

Migration to the cloud

FTC aims to transition all its systems to the cloud by 2030.

Customer created files

Customers need their newly formatted Excel files to be loaded and integrated into their environments within a two-week timeframe.

Training

Due to the absence of a training budget for staff converting the existing mappings, the solution must be intuitive and well-supported with extensive documentation.

Cost savings

The new ETL solution must consolidate all other ETL solutions into a single platform, resulting in cost savings exceeding $1.2 million annually solely in licensing expenses.

Strategic Recommendations from the Fractional Council

Strategic Recommendation #1

After evaluating more than 15 ETL solutions, we recommend adopting Matillion as the preferred choice for FTC. Matillion stands out for several reasons, making it the optimal solution for the needs of FTC :

  • Cloud-Based Integration: Matillion offers seamless integration between on-premises applications and cloud infrastructure, a capability not found in any other existing FTC ETL solutions, including Informatica without substantial costs.
  • Cloud-Optimized ELT Architecture: Matillion's ELT architecture prioritizes speed, delivering faster run times and shorter implementation periods, aligning with our goal of efficient data processing.
  • Migration Support: Matillion enables FTC to streamline their long-term strategy of migrating from Greenplum to Snowflake, facilitating a smooth transition and maximizing the benefits of the chosen data platform.
  • 24x7 Support: Matillion provides comprehensive support, including assistance with application-related queries, ensuring uninterrupted operation and addressing any potential issues promptly.
  • Proven Track Record: Matillion is the preferred ETL solution for numerous Fortune 500 companies, including IBM, Citrix, and Amazon, highlighting its reliability and suitability for enterprise-level deployments.
  • Managed Service Option: Matillion can be installed as a managed service within the FTC AWS environment, offering convenience and optimized performance within our existing infrastructure.

Listen to Tom (CTO), provide more detail around the rational for these recommendation.

Strategic Recommendation #2

To optimize efficiency and cost-effectiveness, it is recommended to migrate all other ETL solutions to Matillion. This consolidation initiative offers the following benefits:

  • Cost Savings: By centralizing ETL operations onto a single solution, an estimated cost reduction of $1.2 million per year can be achieved, primarily by eliminating redundant licensing expenses.
  • Strategic Alignment: Leveraging Matillion aligns with FTC's strategic goal of transitioning all infrastructure to the cloud. Matillion's capabilities bridge the gap between on-premise and cloud infrastructure, supporting our overall cloud migration strategy.
  • Streamlined Development: Consolidating ETL solutions onto Matillion enables streamlined development processes. By reusing and standardizing components, the time-to-market for new ETL development can be significantly reduced, enhancing overall agility and efficiency.

Listen to Doug (CTO), provide more detail around the rational for this recommendation.

Strategic Recommendation #3

Integrate Matillion with Snowflake to drive innovation and generate additional revenue by offering real-time data updates to FTC's customers. The following advantages can be achieved through this integration:

  • CDC Feature: Matillion's Change Data Capture (CDC) feature enables real-time synchronization between on-premise databases and Snowflake. This capability ensures that data updates are immediately reflected in the Snowflake database, providing clients with up-to-date information.
  • Real-time Access: Leveraging Snowflake's schema sharing functionality, clients can have real-time access to data through the integration with Matillion. This capability aligns with customer demands for timely updates and enhances their overall experience.
  • Revenue Generation: FTC's customers have already shown interest in accessing real-time updates via the Snowflake schema share. By delivering this capability, FTC can tap into new revenue opportunities and strengthen customer satisfaction and loyalty.
  • Cloud Migration Support: The integration of Matillion and Snowflake not only generates additional revenue but also supports FTC's cloud migration efforts. It leverages the power and scalability of Snowflake's cloud-based architecture, aligning with FTC's strategic objective of transitioning to the cloud.

Listen to Terry (CSO), provide more detail around the rational for this recommendation.

Cost Estimates

Matillion charges by credit hour and the proposed rate is $2.30 per credit:

(Calculation is # of Hours * 52 *# of cores*2.30)

  • Development – 2 Cores, 1 instance at 80 hours per week ($19,136 per year)

  • QA – 2 Cores, 1 instance at 10 hours per week ($2,392 per year)

  • Production – 4 Cores, 2 Instances (1 Cold) at 60 hours per week ($30,498 per year)Cold instance is calculated as .25 hot cost)

  • Mission Critical Support – 20% of total cost ($7,361 per year)

  • Total estimated Cost $59,444 annually + one time cost of $7,500


The following existing AWS cost will increase based upon increased usage

  • EC2 Instances- None as the existing EC2 intsances will be able to handle the slight increase in volumes
  • S3 Bucket - Under 100.00 per month additional cost(Total file sizes for the month will be under 100GB
  • Snowflake (2.60 per credit hour)- Based upon analysis we expect the additional cost per EC2 instance below:
    • Development – 0 additional hours per week ($0 per year)

    • QA – 10 additional hours(*2 cores) per week ($2,704 per year)

    • Production – 5 additional hours(*4 Cores) per week ($2,704 per year)

    • Total estimated Snowflake cost $5,408 annually

To migrate an existing ETL workflow here are the estimated hours

End of life ETL migration to Matillion -

  • Analysis of 463 ETL jobs - 30 minutes per mapping - Total 231.5 hours
  • Development and QA testing 248 distinct feeds - 2.5 hours per mapping - Total 620 Hours
  • Job Conversion of 463 mappings - 15 minutes per mapping - Total 115.75 hours
  • Total Hour Cost - 967.25

All other ETL tools - TBD

Create new CDC from existing Oracle on-premise DB to Snowflake

Based on a high-level analysis of similar projects undertaken by previous Matillion customers, the estimated effort required to create customer tables and schemas with the ETL component is approximately 500 hours.


Action Plan for FTC

Other Considerations from the Fractional Council

Alignment with Business Strategy: Assess how the adoption of Matillion aligns with the organization's overall business strategy and goals. Evaluate if Matillion supports strategic objectives such as digital transformation, cloud migration, operational efficiency, customer satisfaction, and revenue growth.

Competitive Advantage: Assess how adopting Matillion provides a competitive advantage in the market. Evaluate how Matillion's features, capabilities, and potential for innovation differentiate the organization from competitors.

Customer Satisfaction and Experience: Consider how Matillion's adoption impacts customer satisfaction and experience. Assess if Matillion enhances data accuracy, improves customer service, and provides value to customers.

Implementation and Change Management: Evaluate the feasibility of implementing Matillion and managing the associated change within the organization. Assess the availability of resources, the readiness of the workforce, and the organizational capacity to execute the implementation successfully.

Market Differentiation: Assess how adopting Matillion can differentiate the organization in the market. Evaluate if Matillion's features and capabilities provide a unique selling proposition that can be leveraged in marketing initiatives. Understanding how Matillion's adoption can strengthen the organization's market positioning and competitive advantage is crucial.

Customer Value Proposition: Evaluate how Matillion enhances the organization's customer value proposition. Assess if Matillion's capabilities align with customer needs and if it addresses pain points or challenges faced by customers. Understanding how Matillion can provide value to customers and enhance their overall experience is important in crafting effective marketing messages.

Marketing Campaigns: Consider how Matillion's adoption can be incorporated into marketing campaigns and messaging. Assess if Matillion's benefits can be effectively communicated to target audiences, highlighting the positive impact on data integration, operational efficiency, and business outcomes. Developing marketing strategies and campaigns that effectively promote Matillion's advantages is essential.

Thought Leadership and Content Marketing: Consider how Matillion's adoption can position the organization as a thought leader in the industry. Assess if Matillion's implementation can be used to create valuable content, such as whitepapers, case studies, or webinars, that showcase the organization's expertise in ETL processes and cloud integration. Leveraging Matillion as a topic for content marketing initiatives can enhance brand authority and attract potential customers.

Lead Generation and Conversion: Evaluate how Matillion can contribute to lead generation and conversion. Assess if Matillion's adoption can attract new prospects, generate qualified leads, and contribute to the sales pipeline. Understanding the potential impact on lead generation campaigns, demand generation efforts, and customer acquisition is crucial in assessing the marketing value of Matillion.

Customer Retention and Loyalty: Consider how Matillion's adoption can contribute to customer retention and loyalty. Assess if Matillion's enhanced functionality and improved data integration capabilities can strengthen customer relationships and increase customer satisfaction. Understanding the potential impact on customer retention rates and overall customer loyalty is important in assessing the long-term marketing value of Matillion.

Brand Reputation: Evaluate how Matillion's adoption can impact the organization's brand reputation. Assess if Matillion's association with the organization's marketing initiatives can enhance brand credibility and perception in the market. Understanding the potential positive impact on brand reputation and market positioning is crucial.

By considering these factors, assessment on how Matillion's adoption aligns with the organization's marketing objectives, customer value proposition, marketing campaigns, thought leadership initiatives, lead generation efforts, customer retention strategies, and brand reputation. This evaluation helps determine the marketing value of Matillion and how it can contribute to the organization's marketing success.

Cost Analysis: Carefully evaluate licensing costs, implementation expenses, ongoing maintenance fees, and potential savings achieved through consolidation. Analyze the cost projections and compare them to the estimated cost reductions of $1.2 million per year to determine the financial viability and return on investment (ROI) of the proposal.

Budget Allocation: Analyze availability of funds and budget allocation for the implementation of Matillion. Review the budget priorities, potential reallocation of resources, and the impact on other IT initiatives. Ensure that the necessary financial resources are allocated appropriately to support the adoption of Matillion without compromising other critical business priorities.

Cost Savings: Evaluate the projected cost savings of $1.2 million per year resulting from the consolidation of ETL solutions onto Matillion. Assess the credibility and reliability of these cost-saving estimates, considering factors such as licensing expenses, maintenance costs, and potential efficiency gains. Determine if the expected cost savings align with the organization's financial goals and contribute to improving profitability.

ROI Analysis: Analyze the potential return on investment (ROI) of implementing Matillion. Consider factors such as increased revenue opportunities, improved operational efficiency, reduced time-to-market for new ETL development, and enhanced customer satisfaction. Finally weigh the anticipated benefits against the investment required to determine if the expected ROI justifies the expenditure.

Financial Risk Assessment: Assess the financial risks associated with the proposal. This would involve evaluating factors such as vendor stability, potential cost overruns, budgetary constraints, and the impact on cash flow. Consider the potential risks and develop mitigation strategies to ensure financial stability throughout the implementation and operation of Matillion.

Long-Term Financial Impact: Evaluate the long-term financial impact of adopting Matillion. Assess factors such as scalability, future licensing costs, maintenance fees, and the ability of Matillion to support evolving business needs. Consider the financial sustainability of the solution and its alignment with the organization's long-term strategic objectives.

Compliance and Audit: Consider the compliance and audit aspects related to the adoption of Matillion. Assess if Matillion adheres to regulatory requirements and if it provides the necessary audit trails and controls for data governance and financial reporting. Ensure that the proposed solution meets compliance standards and mitigates any financial risks associated with non-compliance.

This evaluation ensures that the adoption of Matillion aligns with the organization's financial goals and contributes to its overall financial health.

Operational Efficiency: Evaluate how adopting Matillion would enhance operational efficiency. Assess if Matillion streamlines ETL processes, reduces manual efforts, and improves data processing speed. Understanding the potential efficiency gains and their impact on operational productivity would be crucial.

Scalability and Flexibility: Assess if Matillion can scale and adapt to changing business needs. Evaluate the solution's ability to handle increasing data volumes, support new data sources or formats, and accommodate future growth. Scalability and flexibility are vital considerations to ensure that Matillion can effectively meet operational demands.

Integration with Existing Systems: Evaluate how Matillion integrates with existing operational systems and processes. Assess if Matillion can seamlessly integrate with other critical systems (e.g., ERP, CRM) to ensure smooth data flow and interoperability. Understanding the impact on existing operational workflows and the potential for process improvements would be important.

Process Automation: Assess how Matillion enables process automation within the organization. Evaluate if Matillion's features and capabilities can automate repetitive tasks, eliminate manual interventions, and improve process accuracy and reliability. Assessing the potential for process automation and its impact on operational efficiency is crucial.

Time-to-Market: Evaluate how Matillion affects the time-to-market for new ETL development and data integration projects. Assess if Matillion's ease of use, pre-built components, and streamlined development processes can accelerate project timelines and improve speed to market. The ability to quickly deliver data solutions and meet business requirements is a vital consideration.

Risk Management: Consider the risk management aspects associated with adopting Matillion. Assess the reliability and stability of the solution, as well as its ability to handle data security and privacy requirements. Evaluating the risk mitigation strategies, disaster recovery capabilities, and compliance features of Matillion is essential for effective risk management.

Operational Cost Savings: Evaluate the potential cost savings resulting from adopting Matillion. Assess if Matillion reduces operational expenses by eliminating redundant ETL solutions, improving resource utilization, or optimizing data processing. Understanding the potential cost savings and their impact on the organization's financial health is crucial.

This evaluation helps ensure that Matillion aligns with the organization's operational objectives, improves processes, and drives operational excellence.

The recommendation to adopt Matillion as the preferred ETL solution for FTC requires an assessment of the technology infrastructure to ensure a seamless implementation. Here are key considerations:

Cloud Infrastructure Readiness: Matillion is a cloud-based ETL solution, so an assessment of the existing cloud infrastructure is necessary. Evaluate the current cloud provider (e.g., AWS, Azure, Google Cloud) and ensure it aligns with the requirements and capabilities of Matillion. Verify if the necessary resources and configurations are in place to support Matillion's deployment and integration.

On-Premise Integration: Since FTC aims to bridge the gap between on-premise and cloud infrastructure, assess the existing on-premise systems and their compatibility with Matillion. Determine if any modifications or upgrades are required to establish smooth integration between the on-premise and cloud environments.

Data Sources and Connectivity: Evaluate the data sources used by FTC, including databases (e.g., Oracle, SQL Server, Greenplum) and file formats (e.g., Excel, CSV). Ensure that Matillion supports the necessary connectors and APIs to efficiently extract, transform, and load data from these sources. Assess the connectivity options and verify if any additional configuration or setup is needed.

Scalability and Performance: Analyze the anticipated data volumes and processing requirements to assess if the current infrastructure can support Matillion's scalability and performance demands. Consider factors such as processing speed, memory, and storage capacity to ensure the infrastructure can handle the expected workload efficiently.

Security and Compliance: Evaluate the security measures and compliance requirements within the technology infrastructure. Assess if Matillion aligns with the organization's security policies and compliance standards, ensuring data privacy and protection during ETL operations. Review access controls, encryption protocols, and compliance certifications of both Matillion and the cloud infrastructure.

Training and Support: Determine the availability of resources and support for training the staff responsible for the conversion to Matillion. Assess if Matillion provides comprehensive documentation, tutorials, and support channels to facilitate a smooth transition and ongoing usage.

Integration with Existing Tools: Evaluate the compatibility of Matillion with other existing tools and systems utilized by FTC. Ensure that necessary integrations, such as with monitoring or data visualization tools, can be established to provide a cohesive ecosystem.

Disaster Recovery and Business Continuity: Assess the disaster recovery and business continuity capabilities of the technology infrastructure. Verify if Matillion and the cloud infrastructure provide backup mechanisms, redundancy options, and disaster recovery protocols to safeguard data and ensure uninterrupted ETL operations.

By conducting a comprehensive Technology Infrastructure Assessment, FTC can identify any gaps or modifications required to seamlessly implement Matillion as the recommended ETL solution. This assessment ensures that the technology infrastructure is fully equipped to support the functionality, scalability, security, and compliance needs of Matillion, enabling a successful integration and maximizing the benefits of the solution.

Market Demand: Assess the market demand for Matillion and its alignment with customer needs. Analyze the competitive landscape and evaluate if Matillion's features and capabilities provide a competitive advantage. Understanding the market demand and potential customer interest in Matillion would be crucial in determining its sales potential.

Revenue Generation: Evaluate the revenue generation opportunities associated with adopting Matillion. This would include assessing the potential for upselling and cross-selling Matillion to existing customers, as well as identifying new customer segments or industries that would benefit from the solution. Understanding the revenue potential and the expected sales growth would be essential in evaluating the proposal.

Sales Strategy Alignment: Ensure that the proposal aligns with the organization's sales strategy. Assess if Matillion fits into the existing product portfolio and sales approach. Determining how Matillion can be effectively positioned and marketed to customers, as well as evaluating the sales team's readiness to promote and sell the solution, would be crucial.

Customer Success: Consider the impact of adopting Matillion on customer success and satisfaction. Evaluate if Matillion can address customer pain points, deliver the expected outcomes, and provide value to customers. Assessing customer feedback, references, and case studies from similar implementations would help gauge the potential success and customer adoption of Matillion.

Sales Training and Support: Consider the availability of sales training and support from Matillion. Assess if Matillion provides comprehensive training resources, sales enablement materials, and ongoing support to the sales team. Ensuring that the sales team has the necessary knowledge and tools to effectively position and sell Matillion is essential for successful adoption.

Competitive Advantage: Evaluate Matillion's competitive advantage and differentiation in the market. Assess how Matillion compares to other ETL solutions and highlight its unique selling points. Understanding the value proposition of Matillion and how it addresses customer needs better than competitors would be crucial in positioning it effectively in the market.

Sales Targets and ROI: Evaluate the impact of adopting Matillion on sales targets and return on investment (ROI). Consider the sales projections, revenue forecasts, and the time it would take to achieve a positive ROI. Assessing the sales team's ability to meet targets and the potential for Matillion to drive sales growth would be essential considerations.

This evaluation helps determine the viability of the proposal from a sales perspective and ensures alignment with the organization's sales strategy and goals.

Workforce Impact: Assess the impact of adopting Matillion on the workforce. This would include evaluating if any job roles or responsibilities would be affected, and if new skills or training would be required for employees to effectively use Matillion. Understanding the workforce implications and planning for any necessary changes or training programs would be essential.

Talent Acquisition and Retention: Consider how adopting Matillion would impact talent acquisition and retention. Evaluate if Matillion's implementation would require hiring new talent with specific ETL or cloud expertise. Assess how the adoption of a modern ETL solution like Matillion aligns with the organization's talent strategy and the potential impact on employee engagement and retention.

Training and Development: Assess the training and development needs associated with Matillion. Evaluate if Matillion provides comprehensive training resources and if additional training programs or learning initiatives would be required to equip employees with the necessary skills to utilize Matillion effectively. Developing a training plan to ensure employees can adapt to the new ETL solution would be crucial.

Change Management: Consider the change management aspects of implementing Matillion. Assess the readiness of the workforce for change, identify potential resistance, and develop a change management plan to address any challenges. Effective communication, stakeholder engagement, and managing the transition process would be key considerations.

Employee Experience: Evaluate how Matillion would enhance the employee experience. Consider if Matillion's features and capabilities would improve efficiency, collaboration, and productivity. Understanding the potential impact on employee experience and aligning it with the organization's employee-centric approach would be important.


CONCLUSION

In conclusion, the Fractional Council has thoroughly analyzed the ETL challenges faced by FTC. The strategic brief presents actionable recommendations aimed at resolving these challenges, focusing on bridging the gap between on-premise and cloud infrastructure, handling dynamic Excel files with strict SLAs, and integrating with a Linux-based scheduler.

The objective is to provide a cost-effective solution that fulfills client requirements. FTC seeks a new ETL solution that replaces their outdated tool, aligns with their technology roadmap, enhances technical capabilities, meets SLAs, and improves agility.

After evaluating more than 15 ETL solutions, Matillion emerges as the recommended choice due to its exceptional features. Consolidating all other ETL solutions onto Matillion is also advised to optimize efficiency and reduce costs, resulting in substantial savings of $1.2 million annually. Integrating Matillion with Snowflake enables real-time data updates, delivering immediate synchronization, real-time access, revenue generation, and facilitating cloud migration efforts.

Fractional Council Contributers

Christe

CEO/CHRO

Terry

CMO

Ross

CTO

Rich

CFO

Sarah

CSO

Want help making this plan become reality?

What you just experienced is the power of having strategic leadership inside of FTC

We would love the opportunity to talk about the ways that you can partner with the Fractional Council to ensure that this plan is executed properly.

Book a time to chat