What are WANdisco LiveData Cloud Services
LiveData Cloud Services is WANdisco’s portfolio of products that enable petabyte scale cloud data migration as well as active-active data replication to create a LiveData environment where data is always available, accurate, and consistent. With zero downtime and zero data loss, WANdisco LiveData Cloud Services keep geographically dispersed data at any scale consistent between on-premises and cloud environments allowing businesses to operate seamlessly in a hybrid or multi cloud environment.
WANdisco LiveData Platform forms the foundation for WANdisco LiveData Cloud Services, and is based on WANdisco's patented Distributed Coordination Engine, which uses consensus to keep Hadoop and object store data accessible, accurate, and consistent in different locations and across any environment—on-premises, hybrid cloud, multi-region, or multi cloud.
LiveData Platform consists of a set of core functions as well as a plugin architecture and prebuilt set of plugins for extensibility.
- Hadoop & Object Storage Connectivity — support for all major Hadoop distributions and cloud storage
- Coordination Engine — patented coordination engine that uses consensus to ensure data consistency
- Security — compatibility with common security protocols–Kerberos, SSL/TLS, LDAP
- Interfaces/APIs — comprehensive command-line interface, and self-documenting REST API to manage migrations
- Web UI — optional management available via browser-based user interface
- Hive Plugin — replicate Hive metadata as it changes in any cluster, ensuring consistency across all environments
- Ranger Plugin — replicate Apache Ranger policy definitions across multiple environments
- Sentry Plugin — coordinate and replicate Sentry policies across clusters to maintain common policy enforcement
- Backup Plugin — adds backup and data restoration capabilities to LiveData Platform
Three phases of cloud adoption
LiveData Cloud Services are aligned with the three typical phases of cloud adoption, from initial cloud data migration to hybrid cloud and multi cloud.
Cloud Data Migration
- Data first strategy to cloud migration
- Single pass, non-blocking migration with guaranteed consistency
- No business disruption
- Maximize target ingest rates
- Data available as soon as it is migrated
- Ensure consistency between on-premises and cloud environments
- Avoid vendor lock-in
- Gain true data availability
- Reduce cost structures
LiveData for MultiCloud
- Easy management of multiple cloud storage providers
- No vendor lock-in
- Share data across regions and clouds for different applications
- True distributed computing capabilities
Live Data Cloud Services differentiation
WANdisco LiveData Cloud Services are the only set of services able to migrate petabyte scale data lakes to the cloud even as the data is actively changing. While other approaches may be able to migrate large amounts of static data, or small scale environments with changing data, none of the approaches are able to efficiently handle both parameters without introducing significant business risks and costs.
Lift and Shift
A lift and shift approach is used to migrate applications and data from one environment to another with zero or minimal changes. The approach is simplistic and risk laden because of the assumption that no changes are needed. As a result, a majority of these projects fail. Instead of gaining efficiencies promised by the cloud, they merely end up transferring their existing inefficiencies to a new location.
Summary of issues:
- Simplistic and risk laden approach
- Results in failed projects
- Requires downtime during migration to prevent data changes from occurring
- All applications must be cut-over at one time requiring a “big-bang” approach
- Does not take advantage of new capabilities available in target environment
- Often results in unexpected costs and performance issues
An incremental copy approach is where new and modified data are periodically copied from the source to target environment utilizing multiple passes of the source data. The approach requires that all original data from the source system have been migrated to the target, and incremental changes to the data are processed with each subsequent pass. The issue with this approach is that if there is a large volume of data with changes occurring it may be impossible to ever catchup and complete the migration without requiring downtime.
Summary of issues:
- Requires multiple passes of the source data
- May be impossible to catchup with all data changes
- May require application downtime
- Typically only supports small data volumes
Dual Pipeline / Ingest
A dual pipeline or dual ingest approach is where new data is ingested simultaneously into both the source and target environments. This approach requires significant effort to develop, test, operate and maintain the multiple pipelines. It requires that all applications are modified to always update both source and target environments when performing any data changes.
Summary of issues:
- Increased development, test and maintenance efforts
- Impacts to application complexity and performance
- Does not address initial migration of source data
WANdisco LiveData Strategy
WANdisco’s LiveData strategy solves all of the challenges associated with the other migration approaches. With a LiveData strategy migrations can be performed without requiring any application changes or business disruption even while data sets are under active change. WANdisco’s LiveData capabilities perform migrations of any scale with a single pass of the source data, while supporting continuous replication of ongoing changes from source to target. Furthermore, WANdisco’s patented Distributed Coordination Engine enables active-active replication to support true hybrid cloud or multi cloud implementations and ensure 100% data consistency across a distributed environment.
WANdisco’s LiveData approach is able to support data migration and replication use cases regardless of the data volumes or amount of data change occurring, and is the only approach that is able to manage large scale data migrations even with large amounts of changes occurring.