Make an iOS appstore out of Gitlab Pages

Gitlab is evolving more and more to be a “one solution” for many tasks, it started as a github clone service to be installed on premise, now it’s focusing on CI/CD. What once could be a github/gitlab/??? + jenkins setup now can be accomplished just by gitlab itself. In my environment we have an Apple Enterprise Subscription and several suppliers developing iOS apps which we distribute with OTA. Certificates to sign the apps are not shared with suppliers so we have, for every app update, to sign the app ourselves before distributing to the final users (in our case through MDM, being apps for company employee use only). Suppliers, our team and some key-users of the company needed a quick way to install test versions of the apps properly signed. There are plenty of solutions to reach this goal of distributing beta to users (Fabric, TestFlight) etc.), but in our context we needed an internal solution, because devices managed by MDM were subject to some restrictions which don’t play well with those kind of services.

Small digression on OTA distribution OTA distribution is quite simple, in the past was a nightmare but now xcode handles quite well al the things. When you build the ipa for Enterprise or AdHoc distribution with xcode, you can ask it to produce a manifest file. The manifest file is a plist (an xml) which contains some information on the app, version, bundle id, an icon and the link where to retrieve the ipa file. To create an install link usable from iOS devices by users, you have to use a custom apple scheme pointing to that manifest plist:

itms-services://?action=download-manifest&url=https://yourdomainname.com/app.plist

note that:

• the manifest file MUST be served under an https connection
• the webserver must be able to serve these MIME type .ipa application/octet-steam and .plist text/xml

The goal was to streamline an automation process that enables someone to push code to the gitlab repository and then go, from an iOS device, to an HTML page and directly install the updated app.

First setup

The first setup was made with gitlab + Jenkins + a static HTML site generator. The workflow, slightly different for some apps with different level of automation, was something like this:

• a supplier pushes to gitlab new changes
• a build Job on jenkins is triggered which builds the app, signs and moves the produced ipa on a shared location (accessible by other jobs)
• a publish job on jenkins is triggered manually or by the previous job which, with a mix of scripts and a static site generator (we used punch just because it uses json files as input for content), produce the final HTML ~AppStore with all Apps pages and correct manifest.plist to handly install them on the devices.

This approach was good in terms that it did what it had to, but was quite unmaintenable involving several pieces of software completely separated and running on different machines: gitlab to host the sources, jenkins on a OSX machine to build and sign, another machine with a webserver to serve the final website. Also it was involving a lot of “magic by conventions” regarding configurations, scripts and file paths.

Mid setup

When gitlab started to integrate CI/CD we changed this workflow removing Jenkins from the equation and leveraging other gitlab features like the API and its Oauth provider. The software stack was down to gitlab, a simple gitlab runner and a webapp made in python with Flask. The new workflow was:

• push new code to gitlab
• configured gitlab CI is triggered and runner builds and stores the signed ipa as an artifact on gitlab.

No publish step were required because everything else was handled by the flask webapp.

The webapp was not so trivial:

• it uses the oauth features of gitlab to authenticate people that already have access to gitlab and obtain a user token to be used with gitlab API
• a call to the API with the obtained token retrieves the list of projects visible to the user (filering out the non-iOS ones, to be more specific)
• going to a specific project page, again through the gitlab API, we retrieve the artifacts of the latest succesful builds
• we read inside the ipa and generate on the fly a manifest file for the install link (we could have used the manifest generated by xcode and save it along the ipa as artifact)

This solution was better since we removed a piece of software, but we still had 3 machines to run everything (one for gitlab, an osx for the runner, another one for the webapp). We removed the “magic by conventions” and all the scripts, embedding the whole logic within the webapp. But we had to do several tricks (aka bad things) to workaround some gitlab API quirks that made the flask app hard to maintain and quite ugly overall.

Final Setup

When gitlab shipped Pages on the Community Edition, we decided to refactor all the things once again. With this setup we could totally kill the annoying flask app, and remove a machine. I will not go inside how gitlab pages works but long story short you can add a job on a CI configuration to publish a folder as a static site, the hostname at which the site is served depends on the repository you configured the CI job. Go through the documentation for more details.

The new flow now goes like this:

• push new code to gitlab
• configured gitlab CI is triggered and the runner build and store the signed ipa and the manifest file as an artifact on gitlab.
• a pages job produces the HTML static files to be published under the project page along with the previous job artifacts (ipa and manifest).

The last step could be accomplished in several ways, you can use a full fledged static site generator or just a simple script to generate a single html page.

Based on what you want to accomplish you could have a namespace site that lists all the projects you want to expose, and then have single project site with the link to install the build (again RTFM if you don't know what I mean).

A simple .gitlab-ci.yml for this setup could be like this:

stages:  - build  - publishbuild_production:  stage: build  script:    - BASE_URL=https://namespace.yourdomain.com/projectname APPNAME=mightyapp fastlane gym  artifacts:    paths:      - public/*.ipa      - public/*.plist    expire_in: 1 day // we don't need to save this since it is persisted by the pages job laterpages:  stage: publish  script:  - python publish.py "https://namespace.yourdomain.com/projectname"  artifacts:    paths:    - public

I suggest you to use Fastlane gym instead of xcodebuild, to simplify building the ipa and producing the manifest file.

Here is a minimal Gymfile :

output_directory "./public"export_options(  method: "ad-hoc",   manifest: {    appURL: "#{ENV["BASE_URL"]}/#{ENV["APPNAME"]}.ipa",    displayImageURL: "https://placehold.it/600x600",    fullSizeImageURL: "https://placehold.it/600x600"  })output_name ENV["APPNAME"]

Finally this could be a super quick & dirty but working script for generating an html page:

# -*- coding: utf-8 -*-"""<!doctype html><html lang="en"><head>    <meta charset="UTF-8">    <meta name="viewport" content="width=device-width, user-scalable=no, initial-scale=1.0, maximum-scale=1.0, minimum-scale=1.0">    <title>Install {title}</title></head><body>    <div class="main" style="margin:50px; text-align:center;">        <h1>{title}</h1>        <p>{bundleid}</p>        <p>{version}</p>        <a href="itms-services://?action=download-manifest&url={baseurl}/manifest.plist">INSTALL</a>    </div></body></html>"""import sysfrom plistlib import readPlistfrom hashlib import md5from os import walk, environfrom datetime import datetimedef main():    baseurl = sys.argv[1]    plistfile = "public/manifest.plist"    index_html_path = "public/index.html"    plist = readPlist(plistfile)    metadata = plist['items'][0]['metadata']    info = dict()    info["title"] = metadata['title']    info["version"] = metadata['bundle-version']    info["bundleid"] = metadata['bundle-identifier']    info["baseurl"] = baseurl    with open(index_html_path, "w+") as f:        f.write(__doc__.format(**info))if __name__ == '__main__':    main()

The script is really raw and concise, you can go creative by using a template engine or whatever static site generator you want.

Hope to have give you some new idea or at least push you to give gitlab a try ’cause it’s a damned good piece of software!

Full Disclosure: no they don’t pay me unfortunately!

Thanks to Sl3 for proof reading :P

Hacker Noon is how hackers start their afternoons. We’re a part of the @AMIfamily. We are now accepting submissions and happy to discuss advertising & sponsorship opportunities.

To learn more, read our about page, like/message us on Facebook, or simply, tweet/DM @HackerNoon.

If you enjoyed this story, we recommend reading our latest tech stories and trending tech stories. Until next time, don’t take the realities of the world for granted!