How to schedule Docker One-Off Tasks on AWS Fargate using Terraform in 4 steps

1. Create Docker Image

FROM continuumio/miniconda3:4.6.14RUN apt-get updateADD environment.yml /tmp/environment.yml
RUN conda env create -f /tmp/environment.yml
RUN echo "source activate $(head -1 /tmp/environment.yml | cut -d' ' -f2) && conda deactivate" > ~/.bashrcCOPY python ./home/python/ENV PATH /opt/conda/envs/$(head -1 /tmp/environment.yml | cut -d' ' -f2)/bin:$PATH
ENV PYTHONPATH="$PYTHONPATH:/home/python-scripts"
RUN bash -c "source ~/.bashrc"
RUN bash -c "source activate $(head -1 /tmp/environment.yml | cut -d' ' -f2)"
ENTRYPOINT ["python", "home/python/run.py"]
name: my_env
channels:
- conda-forge
dependencies:
- python=3.7
- numpy=1.16.1
- pandas=0.24.1
- psycopg2=2.7.6.1
- sqlalchemy=1.2.18
- pip:
- pg8000==1.13.1
SERVICE_NAME=my_service
IMAGE_TAG="latest"
docker build -t ${SERVICE_NAME}:${IMAGE_TAG} .
docker run -e ENVIRONMENT=local_docker \
--network="host"\
-it -t ${SERVICE_NAME}:${IMAGE_TAG}
import osENVIRONMENT = os.environ["ENVIRONMENT"]print("Docker works in {env}.".format(env=ENVIRONMENT))
Docker works in local_docker.

2. Upload Docker-Image to AWS ECR

AWS_KEY=KEY AWS_SECRET=SECRET AWS_REGION=eu-west-1 LOCAL_PROFILE=my_profile && printf "${AWS_KEY}\n${AWS_SECRET}\n${AWS_REGION}\njson" | aws configure --profile ${LOCAL_PROFILE}
AWS_ACCOUNT_ID=$(aws sts get-caller-identity --output text --query 'Account')
AWS_REGION=$(aws configure get region)
SERVICE_NAME=my_service
IMAGE_TAG="latest"
docker build -t ${SERVICE_NAME}:${IMAGE_TAG} .echo "Logging into AWS ECR..."
loggin=$(aws ecr get-login --region ${AWS_REGION} --no-include-email)
${loggin}
aws ecr create-repository --repository-name ${SERVICE_NAME} \
--region ${AWS_REGION} || echo 'AWS ECR Repository exists: ' \
&& echo '' && aws ecr describe-repositories

echo "Tagging image..."
docker tag $SERVICE_NAME:$IMAGE_TAG $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$SERVICE_NAME:$IMAGE_TAG
echo "Pushing into AWS ECR image repository ${SERVICE_NAME}"
docker push $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/$SERVICE_NAME:$IMAGE_TAG

3. Create Fargate Cluster, Service and Task with Terraform

4. Schedule Task using Cloud Watch Event Rules using Terraform

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store