SHIFT ASIA successfully partnered with a leading global internet services company to execute one of the most complex database migrations in the fintech and eCommerce industry. The project involved migrating over 2 billion user records from legacy Oracle databases to modern Cassandra clusters while maintaining 100% uptime and handling up to 40,000 queries per second (QPS) in live production environments.

The successful migration unified the client’s fragmented ID systems across 70+ business services, enabling seamless user experiences across their global ecosystem while achieving significant cost savings through the transition from expensive Oracle licensing to open-source Cassandra technology.

About the Client

Our client is one of the world’s largest internet service companies, operating more than 70 businesses spanning diverse sectors including eCommerce, travel, digital content, fintech services (credit cards, banking, securities, insurance, electronic money), mobile communications, and professional sports.

The company’s unique strength lies in its integrated ecosystem approach, where all services are interconnected through a unified membership and loyalty program. This ecosystem enables the company’s 100+ million Japan-based members and approximately 2 billion global users to earn and utilize loyalty points across multiple platforms, creating exceptional customer lifetime value and minimizing acquisition costs.

The Challenge: Legacy System Transformation

Core Problem

The client faced a critical infrastructure challenge that threatened the scalability and cost-effectiveness of their global operations. The company’s user identification system was fragmented across different regions, with Japan and global markets operating entirely separate ID systems and databases. This fragmentation created significant operational complexities and limited the company’s ability to deliver seamless cross-regional services.

Project Goals and Strategy

The project was driven by a clear and ambitious strategic vision for the future of the client’s global user base. The core goals were:

• Unify the Platform: Migrate all membership data (both local and global) from the legacy system Oracle to a single unified database platform, Cassandra.
• Ensure Seamless Continuity: To execute the entire migration with minimal possible impact on the hundreds of integrated services and, most importantly, on the end-users.
• Enable Parallel Innovation: To build a stable backend foundation that would allow other critical business projects, such as a company-wide user interface transition to a new “Omni” platform, to proceed in parallel without delay.

Access Management Platform Vision

Create a comprehensive access management platform capable of serving 2 billion users worldwide with the following core pillars:

• Security: Implementation of token-based and passwordless authentication systems
• Usability: Omni-channel self-service capabilities enabling seamless user experiences
• Stability: Reliable and extendable architecture supporting massive scale
• Privacy: 100% compliance with global privacy regulations, including GDPR, CCPA, and PCI DSS standards

Technical Challenges

Legacy Infrastructure Issues:

• Heavy dependence on Oracle databases containing data from before 2000
• Extremely high maintenance and licensing costs
• Multiple software version migrations had created non-standard and unclean data
• Traditional SQL schema structure limits scalability for modern applications

Architectural Complexity:

• Lack of clear separation between business logic and physical data layers
• Multiple database systems across different regions are creating data silos
• Inconsistent data formats and standards across legacy systems

Migration Requirements

The project demanded an unprecedented level of technical precision:

Performance Constraints:

• Handle live production traffic up to 40,000 queries per second
• Maintain zero downtime during the entire migration process
• Ensure complete data integrity across billions of user records

Technical Specifications:

• Migrate from Oracle (traditional SQL with rigid schema) to Cassandra (document-based, schema-less)
• Implement bidirectional data replication during the transition period
• Create fallback mechanisms to Oracle in case of Cassandra issues
• Ensure simultaneous read/write operations on both database systems

SHIFT ASIA’s Solution Architecture

Strategic Approach

SHIFT ASIA designed a comprehensive migration framework that introduced several innovative components to bridge the gap between legacy Oracle systems and modern Cassandra clusters. The solution prioritized business continuity while enabling gradual, controlled migration of user data and services. The architecture was specifically designed to support parallel UI transition to Omni-channel self-service capabilities without disrupting ongoing migration processes.

Key Strategic Elements:

• Zero-impact migration ensuring seamless service continuity
• Parallel development pathways for database migration and UI modernization
• Security-first approach with token-based and passwordless authentication integration
• Compliance-by-design architecture meeting GDPR, CCPA, and PCI DSS requirements

Technical Components

1. Translation Module

• Created a clear separation between business logic and physical data layer
• Enabled seamless communication between different database architectures
• Provided an abstraction layer for consistent data access patterns

2. Migration Controller with Redis Tracking

• Real-time calculation and monitoring of user synchronization status
• Intelligent routing of user requests during migration phases
• Performance optimization through Redis caching mechanisms

3. Migration Trigger with Kafka Integration

• Event-driven architecture for data synchronization
• Real-time listening and publishing of data change events
• Scalable message processing for high-volume operations

4. Bidirectional Replication System

• O2C (Oracle to Cassandra) replicator for forward data migration
• C2O (Cassandra to Oracle) replicator for fallback capabilities
• Continuous data synchronization ensures consistency across both systems

Technology Stack

• Primary Language: Kotlin for robust, scalable application development
• Message Processing: Apache Kafka for high-throughput data streaming
• Caching Layer: Redis for performance optimization and state tracking
• Target Database: Apache Cassandra for modern, distributed data storage
• Source Database: Oracle (legacy system maintenance during transition)
• Security Framework: Token-based and passwordless authentication systems
• Compliance Integration: GDPR, CCPA, and PCI DSS compliant data handling mechanisms

Implementation and Results

Project Execution

The migration was executed by a dedicated team of 20 professionals, equally split between development and quality assurance specialists. This balanced approach ensured both robust implementation and comprehensive testing throughout the 3-year project lifecycle.

Measurable Outcomes

Cost Optimization: The transition from expensive Oracle licensing to open-source Cassandra technology delivered substantial cost savings. The elimination of Oracle licensing fees, combined with reduced maintenance overhead, created significant long-term financial benefits for the client’s global operations.

System Unification: Successfully consolidated fragmented ID systems into a single, unified platform supporting the client’s vision of “One Platform, One ID.” This unification enables seamless user experiences across all 70+ business services and supports the company’s global expansion strategy. The migration created a foundation for the parallel implementation of Omni-channel self-service capabilities.

Security and Compliance Achievement: Implemented a modern access management platform featuring token-based and passwordless authentication systems. The new platform achieved 100% compliance with major global privacy regulations, including GDPR, CCPA, and PCI DSS, providing robust security for 2 billion users worldwide.

Enhanced User Experience: The unified platform enabled the deployment of Omni-channel, significantly improving user experience and reducing operational overhead through automated user management processes.

Performance Achievement: Maintained 100% system availability throughout the 3-year migration process while handling peak loads of 40,000 QPS. The new Cassandra-based system provides improved scalability and performance characteristics compared to the legacy Oracle infrastructure.

Data Integrity: Successfully migrated over 2 billion user accounts with zero data loss or corruption, despite the complexity of converting decades of legacy data across multiple schema formats.

Service Highlights

Technical Excellence

SHIFT ASIA demonstrated exceptional expertise in managing complex database migrations, particularly in handling the architectural differences between traditional SQL databases and modern NoSQL systems. The team’s proficiency in Kotlin development enabled the creation of robust, maintainable migration tools that operated reliably under extreme load conditions.

Innovative Problem Solving

The design and implementation of bidirectional replication systems showcased an advanced understanding of distributed systems architecture. The solution’s ability to maintain consistency across two fundamentally different database technologies while supporting live production traffic represents a significant technical achievement.

Risk Management

The comprehensive fallback mechanisms and real-time monitoring capabilities ensured business continuity throughout the migration process. The Redis-based tracking system provided unprecedented visibility into migration status, enabling proactive issue resolution and optimal resource allocation.

Consultancy and Strategic Support

Beyond technical implementation, SHIFT ASIA provided strategic guidance on technology stack selection, architecture design, and long-term maintenance considerations. The team’s recommendations on Cassandra optimization and Kafka configuration contributed to the solution’s long-term success and scalability.

Why SHIFT ASIA Was Selected

The client chose SHIFT ASIA as their migration partner based on several key factors:

Technical Expertise: SHIFT ASIA’s development team demonstrated deep knowledge of both legacy Oracle systems and modern distributed database technologies. Their proficiency in Kotlin and experience with high-scale Kafka implementations made them the ideal partner for this complex migration.

Proven Track Record: The team’s previous experience with large-scale database migrations and understanding of fintech/eCommerce requirements provided confidence in project success.

Cultural and Communication Fit: Strong collaboration capabilities and clear communication throughout the project lifecycle ensured smooth coordination across the client’s global teams.

Conclusion

The Global ID Unification project represents a landmark achievement in large-scale database migration and system modernization. SHIFT ASIA‘s successful execution of this complex 3-year initiative demonstrates exceptional technical capability and strategic thinking in addressing enterprise-level infrastructure challenges.

The project’s success has positioned the client for continued global expansion while providing a modern, cost-effective infrastructure foundation. The ID system now supports seamless user experiences across the client’s diverse business portfolio, directly contributing to the company’s ecosystem strategy and long-term growth objectives.

This case study exemplifies SHIFT ASIA’s ability to deliver transformational technology solutions that drive business value while managing complex technical and operational requirements in high-stakes enterprise environments.