Posts

Github Add/Update/Delete File(s) on Branch with GraphQL

In input contents is base64 encoded and expectedHeadOid must match with last commit on branch referenced and operation will fails on mismatch: { "input": { "branch": { "repositoryNameWithOwner": "gliptak/githubapitest1", "branchName": "my_branch" }, "message": { "headline": "Hello from GraphQL!" }, "fileChanges": { "additions": [ { "path": "graphql.txt", "contents": "SGVsbG8gZnJvbSBHcmFwaFFMIQo=" } ], "deletions": [ { "path": "README.md" } ] }, "expectedHeadOid": "74512c01ecaec77e38497d448d391065c07ce973" } } Mutation below updates branch with fileChanges listed: mutation ($input: CreateCommitOnBranchInput!

Github Create Branch with GraphQL

First gather some context: { repository(owner: "gliptak", name: "githubapitest1") { id defaultBranchRef { name target { oid } } } } Substitute repositoryId and oid from above and use the name of the to be created branch: { "input": { "clientMutationId": "1234", "name": "refs/heads/<BRANCH_NAME>", "oid": "<OID>", "repositoryId": "<REPOSITORY_ID" } } mutation AddBranch($input: CreateRefInput!) { createRef(input: $input) { ref { name } clientMutationId } } Success returns { "data": { "createRef": { "ref": { "name": "my_branch" }, "clientMutationId": "1234" } } } And in case of error, relevant details are returned

Github List Files Recursively with GraphQL

At this point GraphQL doesn’t seem to provide the right constructs to list files folders of unconstrained depth. Instead, users will make multiple calls into the API. So let’s get started: { "owner": "gliptak", "name": "aws-cdk-python", "rev": "" } query DefaultBranch($owner: String!, $name: String!) { repository(owner: $owner, name: $name) { name defaultBranchRef { name } } } returns { "data": { "repository": { "name": "aws-cdk-python", "defaultBranchRef": { "name": "master" } } } } Subtitute the default branch from above into the query variables with : appended (master: for our example or other syntax HEAD:) and run:

Github List Repositories Paginated with GraphQL

One of the Github GraphQL operations supporting pagination is searching/listing repositories. Below queries utilize query variables running in GitHub’s GraphQL Explorer. { "query": "org:gliptak is:all", "pageSzie": 2, "after": "" } Below setup has 2 for pagination size for demonstration purposes, larger window to be used in real queries. The initial query: query ListRepos($query: String!, $pageSize: Int!) { search(query: $query, type: REPOSITORY, first: $pageSize) { edges { node { ... on Repository { name defaultBranchRef { name } } } } pageInfo { startCursor endCursor hasNextPage } } } returns repo details and cursor values:

Docker Compose Tailing of Log Files

Docker Compose Logs docker compose logs aggregates stdout of containers started with no out-of-the-box capability for aggregating additional logs generated on container filesystems. Implementing sidecar allows tailing logs generated on container volumes. docker-compose.yaml services: counter: image: busybox command: | /bin/sh -c 'i=0; while true; do echo "$$(date) WARN $$i" >> /var/log/1.log; echo "$$(date) INFO $$i" >> /var/log/2.log; i=$$((i+1)); sleep 5; done' volumes: - log-volume:/var/log counter-log: image: busybox command: | /bin/sh -c 'tail -n+1 -F /var/log/*.

Github Create Pull Request with GraphQL

GraphQL API allows for query and mutation (update) operations which can be executed in GitHub’s GraphQL Explorer. Below example creates a PR. First, find the repository id: { repository(owner: "gliptak", name: "githubapitest1") { id } } The repository id returned to be used to create the pull request: { "data": { "repository": { "id": "R_kgDOGS1otA" } } } The mutation (update) creates a new PR from an existing branch (additional details could have been submitted):

Github GraphQL Example

While Github GraphQL offers detailed documentation and an API Explorer, it doesn’t have (Python) code samples. Below is a basic GQL example. The query can be developed/tested in Explorer from gql import gql, Client from gql.transport.requests import RequestsHTTPTransport import click @click.command() @click.option('--githubtoken', envvar="GITHUB_TOKEN", prompt=True, hide_input=True, help='Github token') @click.option('--owner', help='Repository owner', required=True) @click.option('--repository', help='Repository name', required=True) def gh_query_repo(githubtoken, owner, repository): transport=RequestsHTTPTransport( url='https://api.github.com/graphql', use_json=True, headers={ "Authorization": f"Bearer {githubtoken}", }, #verify=False, retries=3, ) client = Client( transport=transport, fetch_schema_from_transport=True, ) query_repo = ''' query Repository($owner: String!

AWS CloudFormation Starter/`null` Template

If the CloudFormation create has an error, the CloudFormation doesn’t roll back so it has to be deleted and recreated. To avoid multiple initial create/delete, the best approach is to use a null CloudFormation. An example is below: AWSTemplateFormatVersion: 2010-09-09 Description: CloudFormation Starter Conditions: Never: !Equals [ a, b ] Resources: NullResource: Type: Custom::Null Condition: Never

Packaging AWS Lambda functions with large dependencies as containers

While the actual application code for many lambda functions might be small, in many (data science) usecases libraries used and to be packaged together push over the deployment package limit of 50 MB compressed. Container packaging to the rescue. A sample Dockerfile installing SageMaker Python SDK ARG WORK_DIR="/home/app/" FROM public.ecr.aws/lambda/python:3.8 ARG WORK_DIR RUN pip install wheel sagemaker COPY main.py . WORKDIR ${WORK_DIR} CMD ["main.handler"] and corresponding main.py import sagemaker import sys def handler(event, context): print(sys.

Running Jupyterhub Docker

Jupyterhub doesn’t seem to have complete instructions available on simply running an instance under Docker with a number of issues, Stackoverflow questions asking for clarifications. These are my quick notes: As the Docker image name indicates, this is singleuser instance preconfigured for user jovyan. Run below command in directory with your notebooks docker run --rm -p 8000:8000 -d -v `pwd`:/home/jovyan/work --name jupyterhub jupyterhub/singleuser jupyterhub There doesn’t seem to be (default) password set for jovyan user, so at this point you cannot login into the GUI.