Accessing Private Google Artifact Registry with Poetry: Local and Docker Setup

Written by dmitriikhalezhin | Published 2024/11/28
Tech Story Tags: docker | cicd | cicd-pipelines | google-cloud | poetry-in-docker | containerized-python-apps | google-artifact-registry | secure-docker-builds

TLDRWhen building Python applications with Poetry in a Docker container, we sometimes encounter issues accessing private packages stored in the Google Artifact Registry. To resolve this for local development, we'll configure Google Cloud credentials within Docker using Docker Compose. This will enable secure access to the GAR repository during the build process.via the TL;DR App

When building Python applications with Poetry in a Docker container, we sometimes encounter issues accessing private packages stored in the Google Artifact Registry (GAR). Locally, this challenge arises becauseĀ docker buildĀ cannot directly handle the Google Cloud credentials in the same way as our CI/CD pipeline, where we leverage service accounts and the Kaniko action for secure builds. To resolve this for local development, we'll configure Google Cloud credentials within Docker using Docker Compose, enabling secure access to the GAR repository during the build process.


Non-Containerized

Pre-requirements

  • Python ^3.12
  • Poetry ^1.7.1
  • Google Cloud SDK ^489.0.0
  • Keyring ^24.0.0
  • keyrings.google-artifactregistry-auth ^1.0.0
  • Read access to your GAR Python repository
  • Logged into GCP (gcloud auth application-default login)

Setup

  1. Keyring Setup

    InstallĀ keyringĀ support for Google Artifact Registry:

    pip install keyring
    pip install keyrings.google-artifactregistry-auth
    

  1. Connect Poetry to your GAR repo

    Access to the private repository in the Google Artifact Registry can be managed through Poetry. First, configure a custom source in Poetry for the GAR repository by running:

    poetry source add --priority=explicit <PACKAGE_NAME> https://<REGION>-python.pkg.dev/<PROJECT>/<REGISTRY>/simple
    

Be sure to appendĀ /simpleĀ to the repository URL for compatibility.

Now you can install packages from your private repo:

poetry add --source <PACKAGE_NAME> [email protected]

Notes

In some cases, accessing the repository may require setting an explicit OAuth token for authentication in Poetry. Use the following command to configure this globally in Poetry:

poetry config http-basic.<PACKAGE_NAME> oauth2accesstoken $(gcloud auth print-access-token)

Containerized

Pre-requirements

  • Docker ^20.10
  • Docker Compose ^2.0
  • Google Cloud SDK ^489.0.0
  • Read access to your GAR Python repository
  • Logged into GCP (gcloud auth application-default login)

Setup

  1. Secrets Configuration

    First, define a secret inĀ docker-compose.yamlĀ using the local path to your credentials file:

    secrets:
      gcloud_credentials:
        file: ~/.config/gcloud/application_default_credentials.json
    

    • We defineĀ gcloud_credentials.fileĀ as a consistent path for Unix-like environments inĀ docker-compose.yaml:

    • This configuration securely passes the credentials file from your local machine to the build context without exposing sensitive data.

  2. Dockerfile Adjustments

    In the Dockerfile, we handle credentials with the following setup:

    ARG GOOGLE_APPLICATION_CREDENTIALS
    ENV GOOGLE_APPLICATION_CREDENTIALS=${GOOGLE_APPLICATION_CREDENTIALS}
    
    RUN --mount=type=secret,id=gcloud_credentials \
        mkdir -p $(dirname ${GOOGLE_APPLICATION_CREDENTIALS}) && \
        cp /run/secrets/gcloud_credentials ${GOOGLE_APPLICATION_CREDENTIALS}
    

    • --mount=type=secret,id=gcloud_credentials: Securely mounts the credentials during the build process.

    • GOOGLE_APPLICATION_CREDENTIALS: Specifies the credential file's path within the container.

  3. Service Configurations

    Here,Ā gcloud_credentialsĀ is the secret mounted at build time, as specified in the secrets configuration.

    some-service:
      build:
        context: .
        dockerfile: ./Dockerfile.local
        args:
          GOOGLE_APPLICATION_CREDENTIALS: /tmp/application_default_credentials.json
        secrets:
          - gcloud_credentials
    

Usage

Just run:

docker-compose up --build

Conclusion

This approach allows localĀ DockerĀ builds to access private GAR resources securely, ensuring that credentials are handled appropriately and remain protected.


Written by dmitriikhalezhin | I may not be a superhero but I'm a DevOps engineer so close enough.
Published by HackerNoon on 2024/11/28