Resolving the .NET MAUI “VersionCode 1.0 is invalid. It must be an integer value.” error when updating Visual Studio 2022 Preview

I installed the Visual Studio 2022 Preview 6 this evening. I had been using Preview 4. I installed Preview 5, but didn’t have a chance to play with it. I have a simple demo app that I have working with, a basic stopwatch type of app. That app had been created with Preview 4 and it more or less worked fine (unless you counted Mac Catalyst and Windows). After I installed Preview 6, I tried to run the project on Android. It failed to compile with the following error message:

Severity	Code	Description	Project	File	Line	Suppression State
Error	XA0003	VersionCode 1.0 is invalid. It must be an integer value.
Parameter name: VersionCode	StopwatchMaui	....\StopwatchMaui\obj\Debug\net6.0-android\android\AndroidManifest.xml

As a test, I created a new .NET MAUI app from Preview 6. It compiled and ran just fine.

So who now, what now? When I first saw the error, I didn’t pay too much attention to the full path, just the file name. With .NET MAUI, there is an AndroidManifest.xml in the android platform folder.

And we take a look at the file, it’s pretty standard, pretty boring AndroidManifest.xml

<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
	<uses-sdk android:minSdkVersion="21" android:targetSdkVersion="31" />
	<application android:allowBackup="true" android:icon="@mipmap/appicon" android:roundIcon="@mipmap/appicon_round" android:supportsRtl="true"></application>
	<uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
</manifest>

No versionCode there. What’s going on? So I went back and actually read the error message and it was complaining about a version of AndroidManifest.xml located in obj\Debug\net6.0-android. That little fellow looks like this:

<?xml version="1.0" encoding="utf-8"?>
<!--
    This code was generated by a tool.
    It was generated from ....\StopwatchMaui\Platforms\Android\AndroidManifest.xml
    Changes to this file may cause incorrect behavior and will be lost if
    the contents are regenerated.
    -->
<manifest xmlns:android="http://schemas.android.com/apk/res/android" android:versionCode="1.0" package="com.companyname.StopwatchMaui" android:versionName="1.0.0">
  <uses-sdk android:minSdkVersion="21" android:targetSdkVersion="31" />
  <uses-permission android:name="android.permission.INTERNET" />
  <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
  <application android:allowBackup="true" android:icon="@mipmap/appicon" android:roundIcon="@mipmap/appicon_round" android:supportsRtl="true" android:name="crc64c1104ba8f6ea44b3.MainApplication" android:label="StopwatchMaui" android:debuggable="true" android:extractNativeLibs="true">
    <activity android:configChanges="orientation|smallestScreenSize|screenLayout|screenSize|uiMode" android:theme="@style/Maui.SplashTheme" android:name="crc64c1104ba8f6ea44b3.MainActivity" android:exported="true">
      <intent-filter>
        <action android:name="android.intent.action.MAIN" />
        <category android:name="android.intent.category.LAUNCHER" />
      </intent-filter>
    </activity>
    <receiver android:enabled="true" android:exported="false" android:label="Essentials Battery Broadcast Receiver" android:name="crc64192d9de59b079c6d.BatteryBroadcastReceiver" />
    <receiver android:enabled="true" android:exported="false" android:label="Essentials Energy Saver Broadcast Receiver" android:name="crc64192d9de59b079c6d.EnergySaverBroadcastReceiver" />
    <receiver android:enabled="true" android:exported="false" android:label="Essentials Connectivity Broadcast Receiver" android:name="crc64192d9de59b079c6d.ConnectivityBroadcastReceiver" />
    <activity android:configChanges="orientation|screenSize" android:name="crc64192d9de59b079c6d.IntermediateActivity" />
    <provider android:authorities="com.companyname.StopwatchMaui.fileProvider" android:exported="false" android:grantUriPermissions="true" android:name="xamarin.essentials.fileProvider">
      <meta-data android:name="android.support.FILE_PROVIDER_PATHS" android:resource="@xml/xamarin_essentials_fileprovider_file_paths" />
    </provider>
    <activity android:configChanges="orientation|screenSize" android:name="crc64192d9de59b079c6d.WebAuthenticatorIntermediateActivity" />
    <service android:name="crc64396a3fe5f8138e3f.KeepAliveService" />
    <provider android:name="mono.MonoRuntimeProvider" android:exported="false" android:initOrder="1999999999" android:authorities="com.companyname.StopwatchMaui.mono.MonoRuntimeProvider.__mono_init__" />
  </application>
</manifest>

If we look at line 8, we see the culprit

<manifest 
    xmlns:android="http://schemas.android.com/apk/res/android" 
    android:versionCode="1.0" 
    package="com.companyname.StopwatchMaui" 
    android:versionName="1.0.0">

In the new app that was created in Preview 6, the same file had the following line:

<manifest 
    xmlns:android="http://schemas.android.com/apk/res/android" 
    android:versionCode="1" 
    package="com.companyname.StopwatchMaui" 
    android:versionName="1.0.0">

So why is the first one bad and the second one good? In the wacky world of Android, android:versionCode has to have an integer value. This is documented here. So now we know what is the actual error, the next question is why that error occurred.

We can’t just edit the obj\Debug\net6.0-android\AndroidManifest.xml file and call it a day. The next time you rebuild the app, that file gets generated from Platforms\Android\AndroidManifest.xml. And apparently it pulls in information from somewhere else as well.

So I took a look at the .csproj files for the working and non-working apps. In the .csproj file generated by Preview 4, the version information was defined with the following two lines

<!-- Versions -->
<ApplicationVersion>1.0</ApplicationVersion>
<AndroidVersionCode>1</AndroidVersionCode>

With the new project freshly generated by Release 6, the same two lines were now a single line, with the ApplicationVersion now set with an integer value.

<!-- Versions -->
<ApplicationVersion>1</ApplicationVersion>

When I changed the “Versions” lines .csproj to match the single line used in the new .csproj Preview 6, the app compiled and deployed to Android. My best guess is that AndroidVersionCode was being used in Preview 4 and sometime after that, they made the breaking change to ApplicationVersion and jettisoned the AndroidVersionCode setting. It’s a preview release of Visual Studio and they are still baking .NET MAUI. This kind of stuff happens and the end result is a better product.

Controlling a WSL installation of redis-server from the Windows command line

If you like using redis for web site caching and you are writing and testing code locally from Windows, you’ll want to figure out how to run a local instance of redis-server. You have a few options. You can run it from another machine that’s running something vaguely Linux-like or MacOS. You can run it from a Docker container under Windows. Or you can run it directly from Windows SubSystem (WSL) for Linux.

For the last few years, Windows (10, 11, Server 2019) comes with a compatibility layer that lets you run Linux binary executables. The current version is WSL 2, but I’m just going to refer to it as WSL.  If you don’t have installed already, just run the following command from an elevated shell (run as administrator)

wsl --install
Installing WSl via command line
Installing WSl via command line

It will install the bits that you need and then ask you to reboot your machine. For more information about installing and configuring WSL, Microsoft has some really good documentation at Install WSL.

The default Linux distribution for WSL is Ubuntu. After you reboot, you may see a Ubuntu shell for a while as Windows installs the bits that you need. It will churn for a bit and then ask you for a username and password. That password will be your sudo (user root) password. One that is done, you should see something like this.

Final step of installing WSL is configuring the default Linux distribution
Final step of installing WSL is configuring the default Linux distribution

The next thing to do is to install redis-server. We can install redis vis apt-get, but before we do that, we need to update apt-get and remove some of the new install shininess off.

sudo apt-get update
sudo apt-get upgrade

After updating apt-get, you’ll want to upgrade it. Confusing? Sort of. Update is updating the instance of apt-get, upgrade is updating all of the packages that were installed via apt-get. After that has finished, you can install redis-server

sudo apt-get install redis-server

There will be some churn and finally, it be installed. Out of the box, redis will not be running. My personal preference is to only run redis when I’m actually using it for development. So I Iike to start and stop it from the command line. Here are commands that you need to know:

sudo service redis-server status
sudo service redis-server start
sudo service redis-server stop

I’ll start up redis with the service start command, and it will come back with a message that redis is starting. And you can use the service status command to verify that it’s running. The acid test is to connect to redis and see if it’s working. You can use the redis-cli tool to set and get a cache value. You should see something like this.

Running redis-cli from the Ubuntu shell
Running redis-cli from the Ubuntu shell

So now redis-server is running. If you close the shell and open up a new one, it will still be running. If you restart WSL or Windows, then it wont be running. Now you can always pop open an Ubuntu shell and start redis-server, but you can do it from a Windows command line

wsl sudo service redis-server status
Running redis commands from PowerShell
Running redis commands from PowerShell

From the screenshot, you can see that WSL passed along the sudo service command to Ubunto. And because I used sudo, I was prompted for the root password. And it returned the same message that I would see from the Ubuntu shell. Since I only have Ubunto installed, that was the default Linux that received the command. If you have multiple distributions installed, you would use “wsl -d DistributionName”. You can get the names of the installed distributions with the “wsl -l” command. Unlike from the Linux shell, each time I invoke “sudo”, I’m prompted for the password. In the Linux shell, you are prompted just the first time you call sudo in a terminal session.

Having to use the root password over and over again can be tedious on a development box. There is a way around that. You can add a file to the /etc/sudoers.d folder in the Linux distribution and remove the root password requirement for the redis-server service. From the Linux shell do the following

cd /etc/sudoers.d
sudo sh
echo "%sudo ALL=(ALL) NOPASSWD: /usr/sbin/service redis-server *" >> allowed-services
sudo chmod 0440 allowed-services
exit

The first thing we do is make /etc/sudoers.d the current folder. Then we use “sudo sh” to gain root access. You need root access to work with this folder. The echo line basically says that you can invoke sudo with no password for redis-server and writes that setting to a file named allowed-services. The file name allowed-services is arbitrary, I picked it because it made sense to me. There is a file in the sudoers.d folder named README, it will explain what the file name restrictions are.

The chmod 0440 command sets the permissions to read-only for the root account and is required for sudoers.d. This allows us to remove the sudo password requirement for redis-server and only for redis-server. Now we can go back to the Windows shell and run the wsl commands without being prompted.

You can even run the redis-cli tool from powershell…

Redis from PowerShell, no password
Redis from PowerShell, no password

The WIndows Subsystem for Linux is one of the hidden gems for developers. And this gem sparkles…

Bonus Round!

Because I’m lazy, I created shortcuts in my PowerShell profile. I added the following functions:

function redstat {wsl sudo service redis-server status}
function redstart {wsl sudo service redis-server start}

And now I can just check the redis-server status via “redstat”

How to access Apple’s App Connect API from C#, Python, and Go. – Part 4

Welcome to Part 4 of a three part series. Last month, I did a series of posts on how to use Apple’s App Connect API to query the team membership list using C#, Go, and Python. This was code I was actually using myself and after I wrote it, it stopped working. The API calls were returning an HTTP 401 error. But only on Windows, on the Mac the code worked.

That was an odd one to track down. It turned out to be a bug with how I was generating the expiration timestamp for the JWT payload.  Apple documents that you can specify up to 20 minutes into the future for the exp value. Don’t do the full 20 minutes. That works most of time. Until it doesn’t. Just set the timeout to 10 minutes into the future and Bob’s your uncle. 

I went back to the repo’s and changed the expiration timeout to 10 minutes for the C#, Go, and Python repos. The C# code was set to 30 minutes, that should have never worked. If you tried the C# code and wondered why it wasn’t working, that was my mistake.

While I was updating the code, I changed the Go version so that you could validate multiple user names by placing them in a line delimited file and pass that file in with a “-userlist” command line parameter.  The code now reads all of the usernames and stores them in an array.  It then gets all of the team members and checks to see if any of the user names in the array match the team member list.  If you need to validate multiple users, this makes it a one and done task.

The “-username” parameter can still be used if you just have one user to check. In that case, it’s treated as an array with just one item in it. If you use both “-username” and “-userlist”, the code will use the user list and ignore the user name passed on the command line.

I’m not sure why the code worked when I originally wrote the posts a couple of months ago. My guess is that Apple used to allow timestamps longer than 20 minutes but made a change that now enforces the expiration to their published specification. Test for edge conditions, but avoid using them.

A quick PowerShell tip

I have a bunch of PowerShell functions that I stick in my $profile file. Simple stuff, things to make my day to day development work easier. With my sieve-like memory, I need a quick way to see the functions. So I wrote a script named “mine.ps1” and it’s basically a tiny help file.  It has stuff like this

write-host Commands -ForegroundColor White
write-host "get-guid-clipboard" -ForegroundColor Yellow
write-host "set-alias lsd get-by-date" -ForegroundColor Yellow
write-host "Set-Alias touch Set-FileTime" -ForegroundColor Yellow

In my $profile, I define those functions. They could (and should) be in a separate file, but I’m lazy. This is part of my PowerShell profile:

function Set-FileTime{
    param(
      [string[]]$paths,
      [bool]$only_modification = $false,
      [bool]$only_access = $false
    )
  
    begin {
      function updateFileSystemInfo([System.IO.FileSystemInfo]$fsInfo) {
        $datetime = get-date
        if ( $only_access )
        {
           $fsInfo.LastAccessTime = $datetime
        }
        elseif ( $only_modification )
        {
           $fsInfo.LastWriteTime = $datetime
        }
        else
        {
           $fsInfo.CreationTime = $datetime
           $fsInfo.LastWriteTime = $datetime
           $fsInfo.LastAccessTime = $datetime
         }
      }
     
      function touchExistingFile($arg) {
        if ($arg -is [System.IO.FileSystemInfo]) {
          updateFileSystemInfo($arg)
        }
        else {
          $resolvedPaths = resolve-path $arg
          foreach ($rpath in $resolvedPaths) {
            if (test-path -type Container $rpath) {
              $fsInfo = new-object System.IO.DirectoryInfo($rpath)
            }
            else {
              $fsInfo = new-object System.IO.FileInfo($rpath)
            }
            updateFileSystemInfo($fsInfo)
          }
        }
      }
     
      function touchNewFile([string]$path) {
        #$null > $path
        Set-Content -Path $path -value $null;
      }
    }
   
    process {
      if ($_) {
        if (test-path $_) {
          touchExistingFile($_)
        }
        else {
          touchNewFile($_)
        }
      }
    }
   
    end {
      if ($paths) {
        foreach ($path in $paths) {
          if (test-path $path) {
            touchExistingFile($path)
          }
          else {
            touchNewFile($path)
          }
        }
      }
    }
  }

function get-by-date {get-childitem | sort LastWriteTime }
function get-guid-clipboard { [guid]::NewGuid() | Set-Clipboard }
set-alias lsd get-by-date
Set-Alias touch Set-FileTime
Set-Alias -Name guidc -Value get-guid-clipboard -Description "Get a GUID and copy it to the clipboard"
function get-mine {. d:\\scripts\mine.ps1}
write-host "Type 'get-mine' for my local functions"
[System.Net.Dns]::GetHostByName($env:computerName).HostName.ToLower()

The touch functions came from the ss64.com site. I end by displaying the current machine name. When you remote into a box of boxes, it’s good to know where you currently are. Now when I fire up a new shell, I’ll see something like this:

PowerShell 7.1.3
Copyright (c) Microsoft Corporation.

https://aka.ms/powershell
Type 'help' to get help.

Type 'get-mine' for my local functions
uberbox
Loading personal and system profiles took 929ms.

How to access Apple’s App Connect API from C#, Python, and Go. – Part 3

In my first post, I wrote about a need to query my company’s membership list from our Apple app store development account. In that post, I used C# to query Apple’s API. Part 2 covered the Python version. For our final installment, we’ll cover how to accomplish the same task with Go.

As with the C# and Python versions, we’ll need to create a signed Javascript Web Token (JWT) and then make some API calls.  I have the code in a Github repository. You can clone it from here. As with the C# project, the code is spread over a couple of modules. 

The code should be portable across platforms. I used Go 1.13.2 on Windows. If you don’t have Go installed, you can grab it from the download page at golang.org. This would be a good place to mention that the only experience that I have with Go is what I learned from writing this applet. I’m pretty comfortable with C#, but everything that I know about Python and Go came from writing this code.

That disclaimer in place, after you install Go, you can grab the code from the repository and follow along. As with the other projects, we use a 3rd party library to generate and sign the Javascript Web Token (JWT). When I wrote the code a couple of months ago, I used dgrijalva’s jwt-go library. It’s no longer being maintained and it recomends to use a community supported clone of that project, golang-jwt/jwt. While writing this blog post, I updated the code to use the community supported project.

If you clone the code from my repo, the jwt library should get downloaded when you build the code. If it doesn’t you can install it the following command
go get github.com/golang-jwt/jwt

Before we run the code, lets take a tour of the code. The entry point is the module IsUserInApple.go. You can follow along with the code from the repo. It will differ from the C# and Python versions by having named command line parameters. We start with the following code

package main

import (
	"flag"
	"fmt"
	"os"
	"path/filepath"
)

func main() {
	// Define command line options
	configPtr := flag.String("config", "./IsUserInApple.json", "Configuration file")
	usernamePtr := flag.String("username", "", "Username to find (in quotes)")

	flag.Parse()

	var userName string = *usernamePtr
	var ConfigFileName string = *configPtr

At line 4, we import the flag module. This module makes it simple to define and read command line parameters. We’ll define 2 parameters, one for the name of the config file (line 12), the second for the name to match (line 13). For our code, we’ll define them as string variables. We pass in the name of the command line parameter, default value, and the help text.

After defining the parameters, we call flag.Parse() to parse the command line parameters that are being passed in. Lines 17-18 assign the flag values to string variables. This code could probably be written with less code, but it works and it’s readable. The next bit of code checks to see if a username was passed. If it doesn’t, it prints the command line help and then quietly dies a good death.

if len(userName) == 0 {
	fmt.Println("Please specify an email address to match (in quotes)")
	flag.PrintDefaults()
	os.Exit(1)
}

The next few lines check to see if a config file was specified.

if len(ConfigFileName) == 0 {
	ex, err := os.Executable()
	if err != nil {
		panic(err)
	}
	ConfigFileName = filepath.Join(filepath.Dir(ex), "IsUserInApple.json")
}

if _, err := os.Stat(ConfigFileName); os.IsNotExist(err) {
	fmt.Println(err)
	os.Exit(2)
}

At line 1, we see if a file has been specified. It hasn’t been specified, then we want to look in the folder that code is running from from for a file named IsUserInApple.json.  The os.Executable() method will return the full path for the running code, or an error message if something went wrong. If that happens, we abandon ship with panic().

If you are new to Go, it’s a little wierd to see both a return value and error message on the left side of a function call. But you get used to it after a while. And it’s a very clean way of getting errors back from a method call. After getting the path, we extract the directory and concatenate the name of the config file.  At line 9, we check to see if the file exists or die trying.

At this point we have a filename for the configuration file and we can attempt to read it.

fmt.Println("Looking for " + userName)

config, err := ReadConfig(ConfigFileName)

Time to dive into the ReadConfig() method. That code is in the AppleJWT.go module. It’s not the cleanest separation of code, but just roll with it. We have a bunch of fun things going on here.  First things first are the imports:

import (
	"crypto/ecdsa"
	"crypto/x509"
	"encoding/json"
	"encoding/pem"
	"fmt"
	"io/ioutil"
	"log"
	"time"

	"github.com/golang-jwt/jwt"
)

The first 8 imports are modules that come with Go. The crypto modules are used for the JWT signing. The encoding modules let us read and parse JSON and PEM data. The last module is the external library that will be pulled directly from Github.

Now we’ll define an object to contain the JSON data from the config file. This is a pretty simple object:

type ConfigSettings struct {
	PrivateKeyFile string `json:"PrivateKeyFile"`
	KeyID          string `json:"KeyID"`
	IssuerID       string `json:"IssuerID"`
}

That will map to the JSON format of the config file, which will look like this

{
    "PrivateKeyFile": "path/to.your/privatekey.p8",
    "KeyID": "ABCDEF1234",
    "IssuerID": "d88b7c23-4c26-48fb-9d62-5649f27a25a2"
}

There are some handy online tools from converting JSON data to Go structs. I have used JSON-to-Go and transform.tools, they both work pretty well. The JSON-to-Go tool gives you the option of using inline type definitions or separate structs for each type. After that we have the ReadConfig method

func ReadConfig(ConfigFileName string) (*ConfigSettings, error) {
	file, err := ioutil.ReadFile(ConfigFileName)

	if err != nil {
		return nil, err
	}

	config := new(ConfigSettings)

	err = json.Unmarshal([]byte(file), &config)

	return config, err
}

Line 1 defines the parameters, the name of the config file, and an error variable. Then we read the file at line 2 with the ReadFile method. If it fails, we return the error. At line 8, we create a new instance of the ConfigSettings struct that will contain our settings. Then at line 10, we use the Unmarshal() method to parse the JSON data into our ConfigSettings struct.

That brings us back to our main method. 

if err != nil {
	fmt.Println(err)
	os.Exit(3)
}

CheckUserList(config, userName)

Skipping over error checking code, we now have the method that will ties this all together for us. Now we can dive in to CheckUserList() defined in AppleStoreApi.go. At the top of module, we have some structs defined.

type AppConnectUsers = struct {
	Data  []Datum              `json:"data"`
	Links AppConnectUsersLinks `json:"links"`
}

type Datum = struct {
	Type       string     `json:"type"`
	Attributes Attributes `json:"attributes"`
}

type Attributes = struct {
	Username  string   `json:"username"`
	FirstName string   `json:"firstName"`
	LastName  string   `json:"lastName"`
	Roles     []string `json:"roles"`
}

type AppConnectUsersLinks = struct {
	Self string `json:"self"`
	Next string `json:"next"`
}

type AppConnectErrors struct {
	Errors []struct {
		Status string `json:"status"`
		Code   string `json:"code"`
		Title  string `json:"title"`
		Detail string `json:"detail"`
	} `json:"errors"`
}

The AppConnectUsers, Datum, Attributes, and AppConnectUsersLinks structs define the JSON data that is returned by the List Users API call. If you look back at Part 1, the JSON document returned by List Users has a lot more in it. We only need to define structs to use the data members that we actually care about. Since this is a read only API call, we can let Unmarshall deserialize only the fields that we will use and it will discard the rest.

We have a second object called AppConnectErrors, with a very different structure. If the API call fails from something happening on Apple’s end, the API will send back a very different JSON document. The following is an example of the JSON that Apple sends back when the API is not available:

{
	"errors": [{
		"status": "500",
		"code": "UNEXPECTED_ERROR",
		"title": "An unexpected error occurred.",
		"detail": "An unexpected error occurred on the server side. If this issue continues, contact us at https://developer.apple.com/contact/."
	}]
}

We’ll cover how to handle that error data later on. The CheckUserList method has a lot going on so we’ll do this in parts.

func CheckUserList(config *ConfigSettings, Username string) {
	token, err := CreateAppleJWT(config)
	if err != nil {
		fmt.Println(err)
	}

	client := &http.Client{}

	var nextUrl string = "https://api.appstoreconnect.apple.com/v1/users?limit=100"

The first thing is that we call CreateAppleJWT in AppleJWT.go to create the signed JWT. Now we’ll jump into that.

func CreateAppleJWT(settings *ConfigSettings) (string, error) {
	bytes, err := ioutil.ReadFile(settings.PrivateKeyFile)

	if err != nil {
		fmt.Println(err)
	}

	x509Encoded, _ := pem.Decode(bytes)

	parsedKey, err := x509.ParsePKCS8PrivateKey(x509Encoded.Bytes)

	if err != nil {
		log.Fatal(err)
	}

	ecdsaPrivateKey, ok := parsedKey.(*ecdsa.PrivateKey)

	if !ok {
		panic("not ecdsa private key")
	}

	token := jwt.NewWithClaims(jwt.SigningMethodES256, jwt.MapClaims{
		"iss": settings.IssuerID,
		"exp": time.Now().Add(time.Minute * 20).Unix(),
		"aud": "appstoreconnect-v1",
	})

	token.Header["kid"] = settings.KeyID

	tokenString, err := token.SignedString(ecdsaPrivateKey)

	if err != nil {
		log.Fatal(err)
	}

	return tokenString, nil
}

At line 2, we read the private key file. Lines 8 through 16 parse the PEM data and come out with the private key in a format that we can use for signing. At line 22, we create the token with an expiration timestamp of 20 minutes into the future. We set the token header kid field to our KeyID. We sign the token at line 30 with our private key and pass it back. Getting the token signed correctly was the hardest part of writing this code. Getting back to CheckUserList…

client := &http.Client{}

var nextUrl string = "https://api.appstoreconnect.apple.com/v1/users?limit=100"

var FoundMatch = false

for {
	req, err := http.NewRequest("GET", nextUrl, nil)
	if err != nil {
		fmt.Print(err.Error())
		os.Exit(3)
	}

	req.Header.Add("Authorization", "Bearer "+token)

	resp, err := client.Do(req)
	if err != nil {
		log.Println("Error on response.\n[ERROR] -", err)
		os.Exit(3)
	}

	defer resp.Body.Close()

	body, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		log.Println("Error while reading the response bytes:", err)
		os.Exit(3)
	}

We start with a new instance of an http Client. We initialize nextUrl with the initial API url. And we set FoundMatch to false. Now we start looping. At line 8, we new up a new request and add our token at line 14. Line 16 is where we call the API and we wait at line 22 to get the response back.

We read the entire body into a byte array at line 24. Now we can start with the parsing. The first thing to do is see if the API returned an error document instead of the list of users.

var appConnectErrors AppConnectErrors
err = json.Unmarshal(body, &appConnectErrors)
if err != nil {
	log.Println("Error while deserializing the response bytes:", err)
	os.Exit(3)
}

// If there is an error object in the body, print it and exit
if len(appConnectErrors.Errors) > 0 {
	firstError := appConnectErrors.Errors[0]
	log.Println("Status:", firstError.Status)
	log.Println("Error accessing API:", firstError.Detail)
	os.Exit(4)
}

We new up an instance the AppConnectError struct as appConnectErrors. Then we unmarshal the body into appConnectErrors. Next we check the length of the Errors array In appConnectErrors. If the API returns users, the Errors object will not be there and the length will be 0.

If the length of Errors is greater than 0, then “Houston, we have a problem“. While Errors is an array and could have more than one error, we don’t care about anything after the first error. So we get the first (and probably only) error. Then we log the error status, which will be a numeric code, and the full text message. And then we die with an error code of 4.

Otherwise we keep going…

var appConnectUsers AppConnectUsers

err = json.Unmarshal(body, &appConnectUsers)
if err != nil {
	log.Println("Error while deserializing the response bytes:", err)
	os.Exit(3)
}

for _, s := range appConnectUsers.Data {
	FoundMatch = strings.EqualFold(s.Attributes.Username, Username)

	if FoundMatch {
		fmt.Printf("Found %s, %s %s, %s\n",
			s.Attributes.Username,
			s.Attributes.FirstName,
			s.Attributes.LastName,
			strings.Join(s.Attributes.Roles, ", "))
		break
	}
}

if FoundMatch {
	break
}

nextUrl = appConnectUsers.Links.Next

if len(nextUrl) == 0 {
	break
}

As with the error struct, we new up a AppConnectUsers struct named appConnectUsers. We unmarshal the JSON data into the struct. At line 9, we iterate through the list of users stored in the Data array. We set FoundMatch at line 10 to true if the username matches. If we have a match, we print that match to the console and use break to exit the for..range loop

Next we break out of the forever loop if we have a match. Otherwise we check to see if we have a next link. If we do, we keep looping, otherwise we are done looping. And we end with an message if we didn’t have a match. 

If you cloned the code from the repo and created your own config file (refer back to Part 1 for that part), you can now compile and run the code. You would compile the applet from your terminal of choice with the following command:

go build .

If you want the smallest executable, build it with the following options:

go build -ldflags="-s -w" .

And then you can compress the resulting executable with a tool like upx. You would need to run IsUserInApple with a required command line parameter, the email to look for. If you leave it out, you will get an error message telling you what is needed. Remember that the IsUserInApple.json file needs to be in the same folder as the executable if you don’t use the second optional command line parameter that will let you specify the name of the config file.

If you are running it with the go command, you would run it as

go run . -username "some.email@company.com"

or

go run . -username "some.email@company.com" -config "IsUserInApple.json"

If you are running it as the compiled executable, you would run it as 

IsUserInApple -username "some.email@company.com

You’ll back something like this:

Looking for some.email@company.com
Found some.email@company.com, Some Email, APP_MANAGER

or….

Looking for some.email@company.com
No match for some.email@company.com

On the rare situation that the API is not available and Apple returns an error document, you’ll see something like this:

Looking for some.email@company.com
2021/07/13 23:26:19 Status: 500
2021/07/13 23:26:19 Error accessing API: An unexpected error occurred on the server side. If this issue continues, contact us at https://developer.apple.com/contact/.
exit status 4

If you run it without  the -username parameter or pass is “-h” or “–help”, you will get back the usuage help messages

go run .                                                                  
Please specify an email address to match (in quotes)
  -config string
        Configuration file (default "./IsUserInApple.json")
  -username string
        Username to find (in quotes)

go run . -h
Usage of C:\Users\anoth\AppData\Local\Temp\go-build2936985627\b001\exe\IsUserInApple.exe:
  -config string
        Configuration file (default "./IsUserInApple.json")
  -username string
        Username to find (in quotes)

That wraps up this series. You can do more than get the team list from the App Connect API. For example, if you want to track how many devices have been provisioned, there is a Devices API. Lots of access points for DevOps functionality.

How to access Apple’s App Connect API from C#, Python, and Go. – Part 2

In my previous post, I wrote about a need to query my company’s membership list from our Apple app store development account. In that post, I used C# to query Apple’s API.  In this installment, we’ll cover how to accomplish the same task with Python.  The final post will cover a Go version.

As with the C# version, we’ll need to create a signed Javascript Web Token (JWT) and then make some API calls.  As with the C# version, I have the code in a Github repository. You can clone it from here. The code is in a script named IsUserInApple.py. You can name your script anything that the OS and Python allows, I’ll be referring to my script as IsUserInApple.py.

I wrote the code using Python 3, version 3.9.2. Unless you are running Windows, the odds are that you have Python 3 installed. If not, you can get it from the good people at python.org. This code should work on any platform that supports Python 3. It should also work (potentially with some changes) with Python 2, but I haven’t tested that. 

This script will use a couple of libraries that you will need to install. To make the HTTPS web requests, we’ll be using the Requests library. It does exactly what it says on the tin, “Requests is an elegant and simple HTTP library for Python, built for human beings”. It makes the code for calling Apple’ API simple and easy to follow.

The other library is Authlib, a library for working with OAuth and OpenID Connect. It has everything we need to create and sign our JWT.  While we need to install all of Authlib, we’ll only being using the jwt module.  To install the libraries, you just need to run the pip command like this:

pip install requests authlib

If you are not familar with pip. it’s the package manager for Python and will be installed when you install Python. If you already have Python installed, then you already have pip installed.

In the folder that you will have the Python script, you will need a configuration file. It performs the same function as the IsUserInApple.json file did in the C# code. It will contain the path to your private key file, the key id, and the issuer id for your account. Please refer back to Part 1 for how to define these values. Instead of using JSON, I used a simple key/value file and it’s named IsUserInApple.config. It should use the following format:

[settings]
private_key = c:/scripts/AuthKey.p8
KEY_ID = ABCDEF1234
ISSUER_ID = d88b7c23-4c26-48fb-9d62-5649f27a25a2

Now we can go through the code. I’m going to jump around a bit, you may want to have the script open in another window. After some comments, we start with the following lines:

import requests, time, json, sys, tempfile, os, configparser
from authlib.jose import jwt

The first line imports the libraries that we’ll need. The requests library was the 3rd party library that we installed via pip, the rest are libraries included with Python. The next line imports the jwt model from the authlib.jose library. The “jose” part of the library name is an acronym for Javascript Object Signing and Encryption.

The next part of the code are the methods we’ll define and use to call the API. We’ll dive into them in a bit. For now, we’ll jump down to the bottom of the script.  We start with

if len(sys.argv) > 1:
    config = configparser.ConfigParser()
    try:
        configPath = os.path.dirname(os.path.abspath(__file__)) + "/IsUserInApple.config"
        config.read(configPath)
    except Exception as e :
        print(str(e))

Line 1 is saying “if we have at least one command line parameter after the name of the script…”. The next line creates an instance of the ConfigParser class, which will allow us to easily read contents from the IsUserInApple.config configuation file. Starting at line 3, we’ll use a try/except block to read the config file. We want to read the config file from the same folder as the script. To get that folder name, we call the abspath method on the __file__ variable. The __file__variable is a “dunder” variable in Python and represents the name of the currently running module. A list of dunder variables can be found in the Python Docs. The dirname method will return the folder part of the file name and then we concatenate that with the name of the config file. And at line 5, we read the config file.

Next, we read the settings into variables with the following code. It’s pretty much self explanatory.

try:
    private_key = config['settings']['private_key']    
    KEY_ID = config['settings']['KEY_ID']    
    ISSUER_ID = config['settings']['ISSUER_ID']    
except Exception as e :
    print(str(e))

Next, we have the following code:

if not os.path.isfile(private_key):
    sys.exit("Error missing private key file for JWT")

UserEmail = sys.argv[1].lower()

print('Looking for a match on ' + UserEmail)

token = getToken(KEY_ID, ISSUER_ID, private_key)

First we check to see if the value provided for private_key is actually a file. Then we set UserEmail to the command line parameter (while converting it to lowercase) and echo that back to the shell. The last line is where we call our getToken method in the script to generate the signed JWT. Now, we’ll jump to that getToken method.

def getToken(KEY_ID, ISSUER_ID, PATH_TO_KEY):
    EXPIRATION_TIME = int(round(time.time() + (20.0 * 60.0))) # 20 minutes timestamp

    with open(PATH_TO_KEY, 'r') as f:
        PRIVATE_KEY = f.read()

    header = {
        "alg": "ES256",
        "kid": KEY_ID,
        "typ": "JWT"
    }

    payload = {
        "iss": ISSUER_ID,
        "exp": EXPIRATION_TIME,
        "aud": "appstoreconnect-v1"
    }

    # Create and return the JWT
    return jwt.encode(header, payload, PRIVATE_KEY)

Line 1 defines the name of the method and the parameter names. No big whoop. Line 2 generates the expiration time, 20 minutes into the future. At line 4, we read in the private key file. The jwt module groks the PEM format, we don’t have to clean it up like we did in the C# code. Then line 7 creates the header and line 13 creates the payload. This is nearly identical to the code from the C# version. At line 20, we create and sign the JWT and return to the code that called it. It follows the same logic as the code in the GetToken method from AppleJWT.cs. Jumping back to where we called getTokem, we have the following line:

members = getAllUsers(token)

Now we’ll dive into the getAllUsers method.

def getAllUsers(token):
    JWT = 'Bearer ' + token.decode()
    URL = 'https://api.appstoreconnect.apple.com/v1/users?limit=100'
    HEAD = {'Authorization': JWT}

    teamMembers = []

    nextURL = URL
    keepGoing = True

    while keepGoing:
        r = requests.get(nextURL, params={}, headers=HEAD)

        y = r.json()

        if 'errors' in y:
            errorCode = y['errors'][0]
            print('Apple returned an HTTP ' + errorCode['status'] + ' code')
            print(errorCode['detail'])
            sys.exit('whoops')

        for i in y['data']:
            teamMembers.append({'username':i['attributes']['username'].lower(), 'roles': ','.join(i['attributes']['roles'])})

        if 'next' in y['links']:
            nextURL = y['links']['next']
        else:
             keepGoing = False

    return teamMembers

Line 1 defines the method name and parameters. Lines 2-4 sets the bearer (token) authorization and the inital URL to call. The next few lines just initialize a few variables and the fun begins at line 11 where we’ll loop until we are done. At line 12, we use the requests library to make a HTTPS get call. Line 14 assigns the JSON results to our Y variable. The next block of code checks the JSON document for “errors”. If Apple’s API falls down, it will fall down this way.

If there are no errors, we just walk through the “data” member of JSON document and add the username and roles for each user to the teamMembers array. When we append the username, we convert it to lowercase. That was we don’t have to worry about case when we match on the email address. Then we check the see if the “links” member of the document has a field named “next”. If it does, we use that as the new URL and keep looping. When we don’t have a “next” url, we return the teamMembers array to caller. That takes us back to the bottom of our script again.

    for i in members:
        if i['username'] == UserEmail:
            HasMatch = True
            print('Match on ' + i['username'] + ', Roles: ' + i['roles'])

    if HasMatch == False:
        print('No match')

else:
    sys.exit("Error: Please specify an email address")

Now we just iterate through the members array and see if the user name matches. We report back on whether we foiund it or not.  The last lines are the error message that would be reported if you ran the script without a email address to match on.

That’s basically all there is to this script. This version doesn’t deserialize the JSON data into objects, it just parses it as is. It’s a trade off. Doing it this way uses less code, but you lose some of the discoverability of having the data in object form.

To run it, you would just do something like:

python IsUserInApple.py some.email@company.com

You’ll either the following if that use is a member of the account:

Looking for a match on some.email@company.com
Match on some.email@company.com, Roles: ADMIN,ACCOUNT_HOLDER

Or the following if that email doesn’t provide a match

Looking for a match on some.email@company.com
No match

Coming up next will be the Go version of this script.


About the image:
This image was derived from some open-source images. The image of the wall comes from Patrick Tomasso via Unsplash. The Apple lock logo comes from Apple, which retains all rights to its artwork. The people icon was created by Monika from the Noun Project.

How to access Apple’s App Connect API from C#, Python, and Go. – Part 1

As the Account Holder for my employer’s Apple App Store account, I get to keep track of who has access to the account. Apple does not do federated logins. Which means no linkage between our Active Directory store and the user accounts associated with the App Store account.

So if someone leaves our company, their Apple Dev account stays active on our team account until we go in and manually remove it. When you have thousands of employees, there’s no way for a developer like me to know who has left the company. And I shouldn’t know that, that’s why we have HR to manage those resources and IT to handle the offboarding.

To make it easier for IT to manage this, I wrote an applet that they could run to see if an employee had an account on on our App Store team. They have a limited API that you can query to get information about your apps and team members. Basically you run that applet and pass in the email address of the employee. It will come back and tell you if that address is a member of the account. 

I had written code in C# that I would run from LINQpad to query various App Connect Services in an ad-hoc manner. I made a stand alone version for our IT department and then realized they would need to have the .NET Framework installed. So I decide to port the functionality to Python and Go (aka “Golang”). Installing the Python runtime is easy and with Go, I can just create a single executable. I’m going to split this blog post into 3 parts, each part covering a different language. This will be the C# version. Part 2 covers Python. The final post in the series, Part 3, covers a Go implementation..

This version of the applet was written for .NET 5. I wrote and tested the code with Windows 10, but it should compile and run under Linux and MacOS as long as the .NET 5 SDK has been installed. All of the source code can be cloned from this repository on Github.

To query Apple for information about your development account, you need to use their App Store Connect API. It’s more or less (I’ve cover that in a bit) documented here.

To authenticate to their API, you’ll need to generate a signed JSON Web Token (JWT). You’ll sign the key with a private key that will be generated on Apple’s site. They’ll hold the public key, you’ll have the private key. This is managed from Apple’s API portal and you can revoke a private key at any time.

You’ll need to generate an API key through your App Store Connect account. The JWT needs to be signed with that key or Apple will reject it.

It’s pretty easy to generate a key. Apple lists the steps here:

  1. Log in to App Store Connect
  2. Select Users and Access, and then select the API Keys tab.
  3. Click Generate API Key or the Add (+) button.
  4. Enter a name for the key. 
  5. Under Access, select the role for the key.
  6. Click Generate.

You’ll be able to download the API private key once as a .p8 file, Apple does not store your private key. You’ll want to store it securely. If you lose it, you’ll have to revoke it and generate a new one.

The API call that we want is List Users. List Users will return all of the users along with their associated metadata. By default, it will the first 100 users, you can increase that up to 200 with the limit parameter. If there are more users than can be supplied in the request, the payload will include a link to get the next set of records. You would keep calling the “next set” URL until it stops including a “next set” URL. 

The JSON shown below is a subset of a fake result set that could be returned by a call to List Users. You get a set of users and the information that we need are fields in the “attributes” object, highlighted below:

{
  "data" : [ {
    "type" : "users",
    "id" : "bded051a-7566-4b5f-a7a9-2e461e51eab0",
    "attributes" : {
      "username" : "smithj@fake.com",
      "firstName" : "John",
      "lastName" : "Smith",
      "roles" : [ "APP_MANAGER" ],
      "allAppsVisible" : false,
      "provisioningAllowed" : true
    },
    "relationships" : {
      "visibleApps" : {
        "links" : {
          "self" : "https://api.appstoreconnect.apple.com/v1/users/bded051a-7566-4b5f-a7a9-2e461e51eab0/relationships/visibleApps",
          "related" : "https://api.appstoreconnect.apple.com/v1/users/bded051a-7566-4b5f-a7a9-2e461e51eab0/visibleApps"
        }
      }
    },
    "links" : {
      "self" : "https://api.appstoreconnect.apple.com/v1/users/bded051a-7566-4b5f-a7a9-2e461e51eab0"
    }
  } ],
  "links" : {
    "self" : "https://api.appstoreconnect.apple.com/v1/users?limit=50",
    "next" : "https://api.appstoreconnect.apple.com/v1/users?cursor=SomeHideousToken&limit=50"
  },
  "meta" : {
    "paging" : {
      "total" : 150,
      "limit" : 50
    }
  }
}

For our purposes, we only care about the email address. We walk through the JSON and just add the address to a list. If there is a “next set” URL, we call that to get the next set of addresses. Then we can compare those emails with the email addresses of the former employees to see which accounts will need to be removed.

This “Next” set URL really isn’t documented as part of the API call. This caused me a fair amount of angst. The API documentation states that the maximum number of users returned will be 200. We have 150+ on the team and I converned about what would happen when we passed 200 members. I emailed Apple Support and was told that API works and I would get all of the members. But they didn’t mention how.

I did some searching in Apple’s forums and the solution was simple and logical. In the JSON result set, there is “links” object. The links object will have a “self” field that contains the URL that was used to make the call. It can have an optional “next” field that will contain a URL that will return the next set of data. When you call the API, you will need to check the “next” field and call that URL until you no longer receive another “next” field in the JSON result set. 

"links" : {
    "self" : "https://api.appstoreconnect.apple.com/v1/users?limit=50",
    "next" : "https://api.appstoreconnect.apple.com/v1/users?cursor=SomeHideousToken&limit=50"
}

Depending on your needs, you can either parse the data in each set or combine it into one set. It (hint) should documented in a way that would be easier to discover. So getting back to the data, we want the “username” from the “attributes” object. The name and role data is nice to have, so we’ll grab it.

"attributes" : {
  "username" : "smithj@fake.com",
  "firstName" : "John",
  "lastName" : "Smith",
  "roles" : [ "APP_MANAGER" ],
  "allAppsVisible" : false,
  "provisioningAllowed" : true
}

The actual API key data is stored in a separate file that I created named IsUserinApple.json. It will contain the path to your private key file, the key id, and the issuer id for your account. It will look something like this:

{
    "PrivateKeyFile": "path/to.your/privatekey.p8",
    "KeyID": "ABCDEF1234",
    "IssuerID": "d88b7c23-4c26-48fb-9d62-5649f27a25a2"
}

That file will not be in the repo, you’ll have to create that one yourself. To get the data, we need to make the API call. We need to pass in a signed JWT. This will be the heaviest lifting for the code. There are three steps:

  1. Create the JWT header
  2. Create the JWT payload
  3. Sign the JWT

For the .NET version, we are going to use a nuget package, jose-jwt, to create and sign the JWT. In the repo for this version of the applet, the code for generated the signed JWT is AppleJWT.cs. The full method is short and we can quickly go over what it does here.

/// <summary>
/// Returns a signed JSON Web Token
/// </summary>
/// <param name="keyId">Your private key ID from App Store Connect</param>
/// <param name="issuerID">Your issuer ID from the API Keys page in App Store Connect</param>
/// <param name="privateKey">The private that was generated by Apple, encoded as Base64</param>
/// <returns>Signed JWT</returns>
public string GetToken(string keyId, string issuerID, string privateKey)
{
    // Create the header
    var header = new Dictionary<string, object>()
    {
        {"alg", "ES256"},
        {"kid", keyId},
        {"typ", "JWT"},
    };

    // Create the payload
    var exp = Math.Round((DateTime.UtcNow.AddMinutes(30) - new DateTime(1970, 1, 1, 0, 0, 0)).TotalSeconds, 0);

    var payload = new Dictionary<string, object>()
    {
        {"aud", "appstoreconnect-v1"},
        { "exp", exp },
        { "iss", issuerID }
    };

    // Generate the signing key from the private that has been given to us
    CngKey key = CngKey.Import(Convert.FromBase64String(privateKey), CngKeyBlobFormat.Pkcs8PrivateBlob);
    
    // Generate the signed token
    string token = Jose.JWT.Encode(payload, key, JwsAlgorithm.ES256, header);

    return token;
}

Lines 11-16 create the header. We set the expiration date at line 19 to 30 minutes into the future with the “exp” field. The issuer ID at line 25 comes from your account in App Store Connect. Apple’s instructions for getting the issuer ID are fairly easy to follow: log in to App Store Connect and: Select Users and Access, then Select the API Keys tab. The issuer ID appears near the top of the page. To copy the issuer ID, click Copy next to the ID. The key ID will also come from that page. It will be the key that we described how to create earlier.

The ID values that you will need

The private key will come from the contents of the .p8 file that you downloaded when you created the API key. The .p8 file will have your private key in PEM format. We generate the signing key from the private key at line 29 and then generate a JWT and sign it with that key at line 32.

There is some code in program.cs to read the PEM data in the .p8 file and return just the private key needed to sign the JWT. The .p8 file will contain something vaguely like this:

-----BEGIN PRIVATE KEY-----
mfykFJXw6fZBxYR4GYDGKyF27GfFaedt4tvR0jACGwUJxBR2t83wZQZOtoj6PMPC
seSrR0XXVWhEzImSXtLfF8JCyHrvpknEgOzSvqD040LUOvgNrnMvrGQHmPGQwmta
So6GaxJuCk0XxJa2t9mjfu5ERvPCIVwQPgeqcVrroNx2qaXzBteLsyonXOPRxo2w
VnPiPup3
-----END PRIVATE KEY-----

Our code needs only what exists between the “BEGIN” and “END” lines. So this code just strips all of that out

private string GetPrivateKey(ConfigSettings configSettings)
{
    var certPEM = File.ReadAllText(configSettings.PrivateKeyFile);

    return certPEM
    .Replace("-----BEGIN PRIVATE KEY-----", "")
    .Replace("\n", "")
    .Replace("\r", "")
    .Replace("-----END PRIVATE KEY-----", "");
}

Now that we have a signed JWT, we can make the API calls. In AppStoreApi.cs, there is a simple wrapper for the List User API call. This unit has a fair amount of code that provides C# models of the JSON data. I used the tools at app.quicktype.io to generate the C# models from the JSON data. I wont repeat that code here, it’s all in repo. But we’ll review the highlights. We start off with by creating a descendant of HttpClient that will be pass the signed JWT in the headers.

public HttpClient client {
    get {
        if (_client == null)
        {
            _client = new HttpClient();    
            _client.DefaultRequestHeaders.Authorization =
                new AuthenticationHeaderValue("Bearer", token);
        }

        return _client;
    }
}

We’ll have a public method, FindUser. It will take an email address and return a user object. Or null, if it doesn’t find a match.

public User FindUser(string EmailAddress)
{
    var users = GetAllUsers();

    if (users != null)
    {
        var user = users.Where(s => s.UserName.Equals(EmailAddress, StringComparison.CurrentCultureIgnoreCase)).FirstOrDefault();

        return user;
    }

    return null;
}

The API call comes in GetAllUsers().

public List<User> GetAllUsers()
{
    List<User> users = new List<User>();

    // Find the first 100 users.  If there there are more than
    // 100 users, the "Next" property will contain the net URL to call
    var jsonString = GetUsers(100, null).Result;

    if (jsonString == null)
    {
        return null;
    }

    var appConnectUsers = AppConnectUsers.FromJson(jsonString);
    
    users.AddRange(appConnectUsers.Data
        .Select(s => s.Attributes)
        .Select(s => new User() {UserName = s.Username, 
                                    LastName = s.LastName, 
                                    FirstName = s.FirstName, 
                                    Roles = s.Roles.ToList()}) );
    
    while (appConnectUsers.Links.Next != null)
    {
        jsonString = GetUsers(100, appConnectUsers.Links.Next.ToString()).Result;
        appConnectUsers = AppConnectUsers.FromJson(jsonString);
        users.AddRange(appConnectUsers.Data
                .Select(s => s.Attributes)
                .Select(s => new User() { UserName = s.Username, 
                                        LastName = s.LastName, 
                                        FirstName = s.FirstName, 
                                        Roles = s.Roles.ToList() }));
    }

    return users;
}

GetUsers is the call the App Connect API and will return a JSON result set. The FromJson method is some boilerplate code for deserializing the JSON. Then we use some LINQ code to iterate through the result and pull the fields from the attributes objects. Then we continue to call the API until we no longer have a Links.Next value.

The GetUsers method is pretty straightforward.

private async Task<string> GetUsers(int count, string nextUrl)
{
    var url = nextUrl ?? $"https://api.appstoreconnect.apple.com/v1/users?limit={count}";
        
    var result = await client.GetAsync(url);
    
    if (result.StatusCode == System.Net.HttpStatusCode.InternalServerError)
    {
        Console.WriteLine(result.StatusCode.ToString());
        return null;
    }
    else
    {
        var users = result.Content.ReadAsStringAsync();

        return users.Result;
    }
}

We pass in null for nextUrl for a new search, otherwise it’s the url for the next set of data. 99.99% of the time you will get a result back. When Apple’s API is down (and I have seen it happen), you can get a 500 error and this code will catch.

Assuming that you have the .NET 5 SDK installed, you can just grab the code from the repo and then build it.  Executing “dotnet build” should pull down the required packages and the build the applet. If you are running it with the dotnet command, you would run it as 

dotnet run -- some.email@company.com

If that email is a member of the account, the applet wiill come back with something line this:

Found user: some.email@company.com, Some Email [AppManager, Developer]

Otherwise something like this:

User: some.email@company.com not found

That’s the core of the code. The rest of the code is some simple stuff for processing the command line parameters and loading the settings. It’s pretty basic stuff and commented in the repo. Our IT staff now uses a version of this code when they process an employee leaving the company. If that person’s email shows up, IT passes the email to me and I can remove that user from the account. We may look at ways to have the applet query the results from the App Connect API and then match all of the emails to see if they exist in our Active Directory. In the meantime, this works.

Coming next will be the Python and Go versions of the code.


About the image:
This image was derived from some open-source images. The image of the wall comes from Patrick Tomasso via Unsplash. The Apple lock logo comes from Apple, which retains all rights to its artwork. The people icon was created by Monika from the Noun Project.

Quick and cheap audio editing for online courses

A family member is a college professor and her courses this fall will be held virtually. Her lectures will be recorded ahead of time and she asked me for some tips to be more productive.

Much of this will be for the tools that she is using, Microsoft Powerpoint and Techsmith Camtasia, but the general ideas work for most tools.

Use a decent microphone. You can go crazy and spend a lot of money on microphones and associated gear, but you can get very good results with a decent USB microphone.

The Blue Yeti microphone is a good choice. It has a stand and you can control the pickup pattern that it uses. The pickup pattern is important, you just want to pick up your voice and nothing else. Amazon.

The HyperX QuadCast microphone is another good choice. Designed for gamers, it has configurable pickup patterns, a built-in pop filter, and a stand. It also comes with a built-in shock mount. That will filter out any sounds that would be transmitted through your desk. Amazon.

The RØDE NT-USB is another good USB microphone that comes with a stand. Amazon. I use its big brother, the RØDE Podcaster. B&H.

You’ll probably want to use a pop filter with your microphone. Put your hand in front of your mouth and then say the letter “L”, followed by the letter “P”. Did you feel more air move when you said “P”?  That’s because the effort to say “P” is a plosive speech sound. Without a pop filter, that rush of air will get picked up by the microphone.

Depending on what comes with the microphone, you may or may not need an external pop filter. You can usually find a decent one that clips to the microphone or stand for $15-$20 dollars.

To get a consistent sound, you’ll want to record in the same place. Every room has different acoustics and it’s jarring to the audience to hear changes in the sound.

You’ll want to deaden the sound, basically remove any echos. If your room has bare walls, your voice will bounce around a bit. I have acoustic panels in my office, they help a lot. You can make your own, it’s basically linen fabric wrapped around insulation, inside a wood frame.  If you are doing a lot of recording, you may want to look into doing that as a DIY project. 

If you can hang up blankets or quilts around you, that will work just as well. And quilts hanging on the wall will look much better than acoustic panels.

The quilting wall in my wife’s home office helps dampen the sound for her Zoom calls.

Get in the habit of speaking into the microphone at the same location and angle. That will help get a consistent sound.

Also, you’ll want to record and edit your audio with a nice set of comfortable headphones. This will allow you to hear exactly what you recorded and filter out everything else that is going on.

I love Camtasia, but the audio editing tools are less evolved than the video editing tools. What I do is to record and edit the audio separately. We’ll be using Audacity here. Audacity is a free and open-source audio editing tool that is available on multiple platforms. You can get it here. Go install it now, I’ll wait.

You can record your audio directly with Audacity or you can take existing audio from a Camtasia project. Either way, you can clean up the audio in less time than it would from within Camtasia.

If you recorded the audio and video together, you can separate the audio and video tracks from within Camtasia. Once you have the clip loaded on the timeline, right-click on the clip and select “Separate Audio and Video”. You’ll now have individual audio and video tracks. 

If you go to the Share menu, you can select “Export Audio Only…”. Save the file as a .wav file. Now you can open the file in Audacity. When you open the file it may give you a warning about choosing an import method. Audacity can either work with a copy of the file or edit it directly. Either way will work.

Warning message from Audacity

Once you have the file opened in Audacity, we can reduce the background noise. Select a section of the file where you are not talking. From the Effect menu, select “Noise Reduction…”.

Selecting the Noise Reduction effect for the selected audio

Once you have the file opened in Audacity, we can reduce the background noise. If you don’t have a lot of background noise, you can skip this step. Select a section of the file where you are not talking. From the Effect menu, select “Noise Reduction…”.

A dialog will open up. Click the “Get Noise Profile” button. It will scan the selected clip and build a noise profile. Then, select the entire clip (press Ctrl-A) and select “Noise Reduction…” from the Effect menu. This time, press the “Ok” button.

The Noise Reduction effect

Now you can normalize the volume in Audacity. This will allow your narration to have the same volume for every clip. You can select Normalize from the effects menu. You set it at a maximum db level. Your peak should be at -6 db. You can go lower, like -8 db, but you once you pick that level, that’s the level you want to use for the entire video. If someone else will be mastering the audio afterward, find out what level they want you to use.

The Normalize effect

And that is the peak level, the overall audio should be in the range from -9db to -18 db. You can use the normalize filter to set the level so that the peak is at -6db. This will raise all of the levels, including pops, clicks, and other noises in the recording in between your words. That makes it easier to find and remove them.

The Hard Limit option of the Limiter effect

If your voice has a lot of dynamic range, you may find that setting the peaks to -6db doesn’t raise the rest of your voice enough. What you can do is to raise the level higher, to say -3db. Then run the hard limiter filter at -6db. That will clip the peaks to -6db and the net effect is that your voice will sounder louder.

Once you have the audio level set, you want to remove any and all sounds in between your words  Not every word, just in the pauses. Select that section so that it’s highlighted and press the Ctrl-L key combination  That will silence that section. You still want to do this after running the noise reduction filter.

To make your time more efficient in Audacity, you can assign keyboard shortcuts to the filters that you used the most. If you open up the Audacity preferences, from the Edit menu or by pressing CTRL-P, you can select “Keyboard” and edit the key bindings. I assigned the CTRL-8, CTRL-9, and CTRL-0 shortcuts to the Noise Reduction, Normalize, and Limiter filters. Those shortcuts were not being used, and they are easy to remember.

Keyboard shortcuts, use them

If you find that your voice has “click” sounds, you’ll want to remove as many of them as you can. There is a de-click effect with Audacity, but I haven’t had much luck with it. I use a program called RX 7 Essentials from Izotrope that does a nice job of removing the clicks out of my voice. It retails for $129, but they run sales from time to time. Two weeks ago, I paid $29 for it. There is 50% education discount on their site. If you are doing a lot of videos, this can save you a lot of time. If you have this tool, you may not even need to use Audacity.

RX Essentials with the De-click dialog

Once you are done cleaning up the audio, you can import it into Camtasia as a media file. Drop it on the timeline in a track above the existing audio track. That will make it easier to line it up in place. Once you have it in the right place, you can just delete the original audio track from the timeline.

If you are doing a presentation that is a series of Powerpoint slides, you’ll want a second of silence in between each slide. Basically, a half-second at the end of one slide, then another half-second at the start of the next slide. You can use the Camtasia tools to measure a 1 one-second clip, but a friend shared a quick way to this. 

Record a one second of silence audio file. Add it to your Camtasia project. At the end of the audio for a slide, add the one-second clip to the timeline. If each slide is it’s one video clip, extend the end of the clip to the halfway position of the one-second clip. Then place the next slide’s clip next to the previous slide on the timeline. If you need a copy of a one-second clip, I put one here that you can download.

If your slides are in a single clip, split the clip at the end of the first slide. Move the timeline cursor to that position and press the “s” key. That will split the clip at that point. You can then drag the of the clip to the halfway point. Make sure to hold the ALT key down when dragging the edge of the clip. That extends the last frame of the clip.

One more tip for Camtasia. Turn off the preferences setting for “Auto-normalize Loudness”. It’s hidden under the Projects tab in the Windows version of Camtasia and defaults to being on for new installations. You’ll want to disable this feature.

If you are following your own script and you have a sentence that you keeping mangling when you try to say, there are a couple of things to try. One is to keep saying it until you get it right. Another thing is to change the wording. What looks good written down may, for whatever reason, fail to flow off your tongue.

Another thing that you do is to pause during the sentence that is killing you. Say part of the sentence, pause, then continue with the rest. You can use the audio editing tools to clip out the pauses so it sounds more natural.

If you need to embed a link in your video and you just have a video file, use a QR code and let them scan the code with their phone. There are tons of free QR code generators on the Internet. I used QRCode Monkey to create the following QR code and placed into a Powerpoint slide.

Scan it with your phone’s camera, it’s just my Twitter timeline

They give you a fair amount of tweaking to the QR code format. If you do something like this, add a 5-second pause and tell people to pause the video while they scan the code with their phone.

And that’s the budget-friendly way of getting a better sound out of your narration.

Using PowerShell to set the Num Lock state

During these work from home times, on the darkest timeline, I spent a lot of my day using my work PC over an RDP connection. For some reason, every time I connected, the Num Lock status would be set to off. Which is silly and annoying. 

When you are connected to a remote machine and you press the Num Lock key on your keyboard, it toggles the Num Lock state for both the client and guest machine. Which can get confusing if you are working with apps on both machines and the Num Lock state is different between the machines. Ideally, you want the Num Lock state to be the same on both machines.

For me, the preference was a command-line option for setting Num Lock. My shell of choice on Windows is PowerShell, so I decided to write a PowerShell script to set the Num Lock Status. The Num Lock key is a toggle switch, press once to turn on, press again to turn off. The script needed to be able to check the current state to be able to set a specified state correctly.

With a little trial and error, I came up with the following script. I put it on gist to make it easier to grab and make your own:

# One parameter, to set the Num Lock state to On or Off, with
# On as the default
Param(
[Parameter(Mandatory=$false)]
[ValidateSet("On", "Off")]
[String[]] $onoff='On'
)
# Get the current state of the Num Lock key
$CurrentState = [console]::NumberLock
# the RequestedState, based in the command line param.
# On is true, Off is false
if ($onoff -eq 'On') {
$RequestedState = $true
}
else {
$RequestedState = $false
}
# If the requested state is the current state, we declare
# victory and go home
if ($RequestedState -eq $CurrentState) {
if ($CurrentState -eq $false)
{
Write-Host 'Num Lock is already off'
}
else {
Write-Host 'Num Lock is already on'
}
}
else {
if ($CurrentState -eq $false)
{
Write-Host 'Num Lock is off, turning on'
}
else {
Write-Host 'Num lock is on, turning off'
}
# If the requested state is not the current state, then
# we need to do a Num Lock press
# Create a new instance of the WScript object and send
# the NumLock key press to it
(New-Object ComObject WScript.Shell).SendKeys('{NUMLOCK}')
}
view raw num-lock.ps1 hosted with ❤ by GitHub

 

I put the comments inline, it should be pretty self-explanatory with the comments. While PowerShell is supported on macOS and Linux now, this is a Windows only script. The [console] ::NumberLock expression returns the following error message:

OperationStopped: Operation is not supported on this platform.

Which is odd, [console] is a shortcut for [system.console], a class that is accessible on the macOS Powershell. If you run the following command in a PowerShell on either Windows or macOS, you’ll get a list of static properties that should be readable from a POSH script.

[system.console] | Get-Member -Static -MemberType property | Format-Table 

And the NumberLock property is listed, but just not implemented. And that’s not even the real sticking point. This script creates an instance of a WScript object and uses it’s SendKeys method to pass in a Num Lock key press. WScript is the Windows Script Host, a technology that lets scripting languages make Windows API calls. A minor sticking point, I only need this Windows. You can get a list of the special keys that SendKeys can send from here.

That’s the heavy lifting. Because I am lazy, I don’t want to type in the name of the script. So I created an alias for it.

Set-Alias nl d:\scripts\num-lock.ps1

Now I can just run “nl” or “nl On” or “nl Off” to change the Num Lock state. I added the alias to the profile so it’s always available. Now when I connect to my work PC, I run “nl” and all is well.

Colored notes in OneNote

I like using OneNote when I’m working on a project with different kinds of assets. I’m in the middle of doing a new course for Pluralsight that will be out in a couple of months. I have slides, a script, PowerPoint decks, code, media files, etc. It can get a little confusing trying to keep track of everything.

For the new course, I have been using OneNote to map out each module. The way I create the content is that I create a PowerPoint deck for each module and then write a script for the narration of the deck. I create a new OneNote file for the course, and then add a tab for each module. Inside each module, there are tabs for each section of that module. I have one tab for just for notes and links to resources.

Each tab has multiple OneNote paragraphs, basically one paragraph for each slide. To make it easy to pick out paragraphs, I wanted to give each paragraph it’s own background colors. It makes it easier to find the text that I want and I can move stuff around and easily see what text belongs in each paragraph. You would think it would be as simple as right-clicking in the paragraph and selecting the color. No, that is not an option.

You could just place a colored shape behind the paragraph. That kind of works, but it’s an annoyance if you are constantly editing the text as the shape will not resize with the text.

There is another way, and it’s a bit of a hack. You can do this for new text or for a paragraph that you already have added.

For an existing paragraph, copy the text, and cut it from the paragraph. Then, insert a 1×1 table into the paragraph. Next, paste the text into the single cell table.

Click on the table and select the Layout tab in the OneNote ribbon. In the Format section of the Layout ribbon, select the drop-down arrow for shading. Now you can pick the color that you want for the paragraph. Now you have colored paragraphs and the color matches the size of the text. And with the default borders for a table, you get a simple box drawn around the text.

For a new paragraph, insert the 1×1 table first and just start typing into the table cell. By default, OneNote will add a new table row for each hard return of the Enter key. A way around that is to press Shift-Enter twice at the end of the first line and arrow back up the first newline. As long as you leave one empty line at the bottom, pressing the Enter key will keep the text in a single row.

I can take no credit for this technique, I saw it mentioned a few times on the Internets.