This document helps you plan, design, and implement the migration of your containers to Google Cloud. If done incorrectly, moving your workloads from one environment to another can be a challenging task, so plan and execute your migration carefully

This document is part of a multi-part series about migrating to Google Cloud. If you're interested in an overview of the series, see Migration to Google Cloud: Choosing your migration path

This document is part of a series that discusses migrating containers to Google Cloud:
- Migrating containers to Google Cloud: Getting started (this document)
- Migrating containers to Google Cloud: Migrating Kubernetes to Google Kubernetes Engine (GKE)
- Migrating containers to Google Cloud: Migrating to a new GKE environment
- Migrating containers to Google Cloud: Migrating to a multi-cluster GKE environment with Multi Cluster Service Discovery and Multi Cluster Ingress
- Migrating containers to Google Cloud: Migrating from OpenShift to Anthos
This document is useful for a wide variety of scenarios—whether you're starting with containers running on-premises, in a private hosting environment, or in another cloud provider, and whether you're migrating your entire workload to Google Cloud or maintaining part of your workload on-premises or in a private hosting environment

This document is also useful if you're evaluating the opportunity to migrate and want to explore what it might look like, and which options you have. There are various environments for a container to run workloads available on Google Cloud. Choosing one option over the others depends on several factors, and no option is inherently better than the others. Each environment has its own strengths and weaknesses. To choose an environment, do the following:
- Establish a set of criteria to evaluate the container environments to run workloads

- Assess each environment against the evaluation criteria

- Choose the environment that best suits your needs

You don't have to choose the same environment for all your workloads. If you have different types or classes of workloads, you can choose different environments for each of those types or classes

## Designing the migration to Google Cloud
To migrate your containers from your source environment to Google Cloud, we recommend that you follow the framework described in the Migration to Google Cloud series

The following diagram illustrates the path of your migration journey


The framework illustrated in the preceding diagram has four phases:
Assess. In this phase, you assess your source environment, assess the workloads that you want to migrate to Google Cloud, and assess which environment can support each workload. Plan. In this phase, you create the basic infrastructure for your workloads, such as provisioning the resource hierarchy and setting up network access. Deploy. In this phase, you migrate the containers from the source environment to Google Cloud. Optimize. In this phase, you begin to take advantage of the cloud technologies and capabilities

## Establishing criteria to evaluate the container environments to run workloads
To establish the criteria to evaluate the options for the container environment to run workloads, you consider the most important features that you need in these environments. To gather information about which features you need the most, you assess your workloads. For more information about assessing your workloads, see Migration to Google Cloud: Assessing and discovering your workloads

These evaluation criteria and the order they're listed in is an example. You should assess your workloads to compile a list of the criteria that are important for you and your workloads, and order them according to importance. For example, after assessing your workloads, you might consider the following evaluation criteria, listed in order of importance:

Performance. Does the environment add overhead that might degrade the performance of your workloads? Scalability. Which scalability features does the environment offer? Are they enough for the scalability requirements of your workloads, both in terms of reaction times and scalability logic? Degree of control and flexibility. How much control do you want over the environment? Can you customize the environment to your needs? Reliability. Which guarantees does the environment offer? Are they enough for your workloads? Is it reliable enough to implement effective high availability and disaster recovery strategies? Management burden. How much effort do you need to spend to manage the environment? Do you need to train your teams to gather the necessary skills or can you use their existing knowledge? Requirements to use the service. Are there any requirements, technical contracts, or interfaces that your workloads have to adhere to? Do you need to spend significant effort to make your workload compatible with the environment? Data persistence. Does the container environment to run workloads support data persistence? Is this persistency compatible with the requirements of your workloads, including performance, reliability, and legal regulations? Pricing model and costs. Can you use the environment in a cost effective way? Are you able to get proper return of investment from switching to a container environment to run workloads? Future proofing. Does the environment offer upgrade paths that you can use to evolve your workloads? Integration with other services. Does the environment integrate with other Google Cloud services and with other cloud providers' services? Lock-in. Does the environment lock you in to particular technologies, paradigms, or interfaces? Is the environment hindering the portability of your workloads? Security. Does the environment meet your security and privacy requirements?
## Assessing the container environment to run workloads
On Google Cloud, you have different options for running containers. To choose the best option for your workloads, you first assess them against the evaluation criteria that you previously established. For each environment, you assign it a score against each evaluation criterion from an arbitrary, ordered scale. For example, you can assign each environment a score from a scale from 1 to 10 against each evaluation criterion

To run containers on Google Cloud, we recommend the following options, which are presented in increasing order of how much control you have on the underlying infrastructure:
- Cloud Run and Cloud Run for Anthos
- Google Kubernetes Engine (GKE) and Anthos clusters
- Compute Engine
You might be able to assign scores against some of the criteria by reading the product documentation. For example, you can already evaluate Cloud Run against the performance, scalability, degree of control and flexibility, integration with other services. However, to assign scores against other criteria, you might need to design and execute more in-depth benchmarks and simulations. For example, you might need to benchmark the performance of different container runtimes to assess whether they add considerable overhead to your workloads

 Cloud Run and Cloud Run for Anthos
Cloud Run is a managed platform to run containerized, stateless workloads that are built on Knative. Containerized workloads managed by Cloud Run can run on the following:
- If you choose Cloud Run, your workloads run on Google-managed infrastructure

- If you choose Cloud Run for Anthos, your workloads run on GKE, which can be on Google Cloud, on-premises, or on other cloud providers

Use the following list to evaluate Cloud Run and Cloud Run for Anthos against the criteria that you established earlier:

Performance. Cloud Run and Cloud Run for Anthos use Docker containers, which have similar performance to non-containerized workloads, so the containers don't add any considerable performance overhead. Scalability. Cloud Run automatically scales the instances of your workloads and lets you scale your application in to zero instances. This capability is useful if your workloads don't need to have instances running all the time. To minimize the instance startup time, optimize your workloads initialization. Degree of control and flexibility. Cloud Run and Cloud Run for Anthos are suitable for workloads that require full control of the containerized environment where your workloads run, but you don't need to customize that environment. Reliability. Cloud Run and Cloud Run for Anthos integrate with Cloud Monitoring, Cloud Logging, Cloud Audit Logs, and Error Reporting so that you have coverage on performance monitoring, and access to container, request, error, and audit logs. Management burden. Cloud Run and Cloud Run for Anthos manage the environment so that you can focus on your workloads, instead of spending effort to provision, configure, and maintain the underlying infrastructure. Requirements to use the service. Your workloads must adhere to a container runtime contract, so if you can't spend additional effort to make them compatible with Cloud Run, we recommend choosing one of the other options. For more information about Cloud Run limitations, see Cloud Run known issues. Data persistence. Cloud Run and Cloud Run for Anthos are designed to run stateless containers. If your workloads have data persistence requirements, you have to provision and configure another data persistence system. If you need a container runtime environment for stateful workloads, we recommend choosing a different option. Pricing model and costs. Cloud Run charges for the computing resources that your workloads use. Cloud Run for Anthos is included in the Anthos subscription. Future proofing. Cloud Run lets you do rollbacks, gradual rollouts, and traffic migration. You can use these features for your deployment pipelines. Integration with other services. Cloud Run can connect to a Virtual Private Cloud (VPC) network that allows access to Compute Engine VMs and any other resources with internal IP addresses. Lock-in. Cloud Run is built on Knative. If you spend effort to make your workloads compatible with Knative, you can run your containerized workloads in Cloud Run, GKE, Anthos clusters on VMware, or any other Knative-compatible runtime environment without any further modification. Security. Workloads running on Cloud Run are sandboxed using gVisor. Cloud Run for Anthos doesn't use any container sandbox, but uses the default Kubernetes container isolation features. You can protect your Cloud Run resources by managing access with Identity and Access Management (IAM) and configuring a service identity
For more information, see Choosing a Cloud Run platform

 GKE and Anthos clusters
GKE and Anthos clusters are Google-managed services that provide a container environment to run workloads. Both GKE and Anthos clusters run your containerized workloads in Kubernetes clusters. With GKE, clusters run on Google Cloud and with Anthos clusters, clusters can run on Google Cloud, on-premises, or in other public cloud environments

Use the following list to evaluate GKE and Anthos clusters against the criteria that you established earlier:
Performance. GKE and Anthos clusters use Docker containers, which have similar performance to non-containerized workloads so the containers don't add any considerable performance overhead. Scalability. GKE and Anthos clusters include fine-tuned scaling logic that you can adapt to your own needs. You can scale your workloads and your GKE and Anthos clusters clusters both vertically and horizontally. If you don't need complex scaling logic, we recommend choosing a different option because otherwise you might need to spend significant effort to configure effective scalability mechanisms. Degree of control and flexibility. You can provision and configure GKE and Anthos clusters clusters according to your needs. You can personalize every aspect of the cluster nodes, including storage, networking, and security. Google manages the control plane for you so if you need to customize the configuration of the control plane, we recommend choosing a different option. Reliability. GKE and Anthos clusters integrate with Cloud Monitoring and Cloud Logging so that you have complete coverage on performance monitoring and access to container, request, error, and audit logs. You can increase the reliability of your environment with GKE regional clusters and Anthos clusters high-availability options. Management burden. With GKE, you don't have to manage the control plane of your clusters, and Anthos clusters lets you manage all Kubernetes clusters with the same toolchain and processes. This feature greatly reduces the effort that you need to spend to manage the environment, but you need to manage part of the underlying infrastructure. For example, with GKE, you can manage cluster nodes. Most of the management operations can be automated, but it's still something you have to consider when planning the effort required to maintain the environment. If you need a fully managed container environment to run workloads, we recommend choosing a different option. Requirements to use the service. To deploy your workloads on GKE or Anthos clusters, you have to containerize them. Data persistence. GKE and Anthos can run stateful applications and persistent disk storage. Pricing model and costs. GKE charges a cluster management fee and for the resources that your cluster nodes use. Anthos clusters is included in the Anthos subscription. Future proofing. GKE and Anthos clusters both have the features to handle complex deployment processes. Integration with other services. The workloads deployed on GKE and Anthos clusters can be granted access to other Google Cloud services, if you set up the necessary connectivity, authentication, and authorization systems. Lock-in. After containerizing your workloads to run them on GKE or Anthos clusters, you can port them to other environments with minor adjustments. Kubernetes is a portable platform and it doesn't lock you into a vendor environment. Security. GKE offers many ways to protect your nodes, control plane, and workloads:
For more information, see GKE security overview

For more information about migrating to GKE and Anthos clusters, see Migrating containers to Google Cloud: Migrating Kubernetes to GKE and Migrating containers to Google Cloud: Migrating from OpenShift to Anthos

 Compute Engine
Compute Engine lets you create and run VMs on Google infrastructure

Though it's possible to run containers on Compute Engine VMs, we recommend that you choose one of the other container environments to run workloads that are described in this document. The effort you need to spend to operate a self-managed environment running on Compute Engine greatly outweighs the benefits you might get in return

However, if you choose to run containers on Compute Engine VMs, use the following list to evaluate Compute Engine against the criteria that you established earlier:

Performance. Compute Engine VMs don't have any container runtime pre-installed, so choose the one that best meets your requirements. Scalability. Compute Engine uses managed instance groups to automatically scale VM instances. When configuring the autoscaling mechanisms of a managed instance group, you define the autoscaling signals that Compute Engine uses to scale the managed instance group in or out. Degree of control and flexibility. You can customize every aspect of the provisioning and configuration of each Compute Engine VM as long as you adhere to Compute Engine resource quotas. Reliability. You can monitor Compute Engine VMs with Cloud Monitoring, Cloud Logging, and Cloud Audit Logs so that you have complete coverage on performance monitoring and logs. Compute Engine also uses managed instance groups, instance health checking, and autohealing. Management burden. Compute VMs are self-managed so plan for a considerable operational effort to effectively manage the environment. Requirements to use the service. As long as your workloads run on one of the supported operating systems, you can run them on Compute Engine VMs. For more information about Compute Engine limitations, see Compute Engine known issues. Data persistence. Compute Engine has different options for data persistence, such as zonal persistent disks, regional persistent disks, and local solid-state drives. Pricing model and costs. Compute Engine charges for the resources that your workloads need. Future proofing. You can install any integration, deployment, provisioning, or configuration management tool on Compute Engine VMs for your deployment processes. Integration with other services. The workloads deployed on Compute Engine can be granted access to other Google Cloud services, if you set up the necessary connectivity, authentication, and authorization systems. Lock-in. By using Compute Engine, you're not locked into any proprietary product or service. You can build your own operating system images so that your provisioning and configuration processes are automated and portable. Security. Compute Engine helps enhance the security of your environment:
- Access control mechanisms

- Shielded VMs to verify the integrity of your VMs

- Sole-tenant nodes to have exclusive access to physical nodes

For more information, see Securing Google Cloud

## Choosing the correct option for your target environment
In the previous sections, you assigned a value to every criteria for each product. To calculate the total score of each container environment to run workloads, you add all the ratings for that environment based on the criteria. For example, if an environment scored 10 against the performance criterion, and 6 against the scalability criterion, the total score of that environment is 16

You can also assign different weights to the score against each criterion so that you can represent the importance that each criterion has for your evaluation. For example, if performance is more important than scalability in your evaluation, you might define multipliers to reflect that: a 1.0 multiplier for performance and a 0.7 multiplier for scalability. You then use these multipliers to calculate the total score of an option

After calculating the total score of each environment that you evaluated, you organize the environments by their total score, in descending order. Then, pick the option with the highest score as your environment of choice

There are multiple ways to represent this data—for example, you can visualize the results with a chart suitable to represent multivariate data, such as a radar chart

## What's next
- Migration to Google Cloud: Getting started
- Migrating containers to Google Cloud: Migrating Kubernetes to GKE
- Migrating containers to Google Cloud: Migrating from OpenShift to Anthos
- Explore reference architectures, diagrams, tutorials, and best practices about Google Cloud. Take a look at our Cloud Architecture Center.