How to get your own API credentialsSource:
Here we describe how to obtain different types of credentials that can be important when working with a Google API:
- API key (not relevant to all APIs)
- OAuth 2.0 client ID and secret
- Service account token
- External account token (“workload identity federation”)
This can be important for both users and developers:
- Package authors: If you are writing a package to wrap a Google API, you may provide some built-in auth assets so that things “just work” for your users. Regardless, you will need credentials to use during package development and in testing.
- Package users: Wrapper packages may or may not provide default auth assets. If they don’t, you are required to provide your own. Even if they do, you may prefer to bring your own, to have more control over your own destiny. With your own credentials, you avoid sharing quota with other users, which can reduce time-consuming errors and retries. If you use your own credentials, you are no longer at the mercy of someone else choosing to roll the credentials when you least expect it.
- Everyone: The best method for auth in non-interactive settings is to use a service account token or workload identity federation, which require some advance setup.
Note that most users of gargle-using packages do not need to read this and can just enjoy the automatic token flow. This article is for people who have a specific reason to be more proactive about auth.
You will need a Google Cloud Platform (GCP) project to hold your credentials.
Go to the Google Cloud Platform Console:
- This may involve logging in or selecting your preferred Google identity.
- This may involve selecting the relevant organization.
This console is your general destination for inspecting and modifying your GCP projects.
Create a new project here, if necessary. Otherwise, select the project of interest, if you have more than one.
Enable the relevant APIs(s) for your GCP project.
In the left sidebar, navigate to APIs & Services > Library.
Identify the API of interest. Click Enable.
If you get this wrong, i.e. need to enable more APIs later, you can always come back and do this then.
For some APIs, you won’t be able to do anything interesting with the credentials hosted in your project unless you have also linked a billing account. This is true, for example, for BigQuery and anything that has to do with Maps. This is NOT true, for example, for Drive or Sheets or Gmail.
If your target API requires a billing account, that obviously raises the stakes for how you manage any API keys, OAuth clients, or service account tokens. Plan accordingly.
If you’re new to Google Cloud Platform, you’ll get to enjoy GCP Free Tier. At the time of writing, this means you get $300 credit and no additional billing will happen without your express consent. So there is a low-stress way to experiment with APIs, with a billing account enabled, without putting actual money on the line.
Some APIs accept requests to read public resources, in which case the request can be sent with an API key in lieu of a token. If this is possible, it’s a good idea to expose this workflow in a wrapper package, because then your users can decide to go into a “de-authed” mode. When using the package in a non-interactive or indirect fashion (e.g. a scheduled job on a remote server or via Shiny), it is wonderful to NOT have to manage a token, if the work can be done with an API key instead.
Some APIs aren’t really usable without a token, in which case an API key may not be relevant and you can ignore this section.
- From the GCP Console, in the target GCP Project, go to APIs & Services > Credentials.
- Do Create credentials > API key.
- You can capture the new API key via clipboard right away or close this pop-up and copy it later from the Credentials page.
- In any case, I suggest you take the opportunity to edit the API key from the Credentials page and give it a nickname.
Package maintainers might want to build an API key in as a fallback, possibly taking some measures to obfuscate the key and limit its use to your package.
Package users could register an API key for use with a wrapper
package. For example, in googlesheets4, one would use
googlesheets4::gs4_auth_configure() to store a key for use
in downstream requests, i.e. after a call to
Most APIs are used to create and modify resources on behalf of the user and these requests must include the user’s token. A regular user will generally need to send an OAuth2 token, which is obtained under the auspices of an OAuth “app” or “client”. This is called three-legged OAuth, where the 3 legs are the client, the user, and Google.
The basic steps are described in the Prerequisites section for doing Google OAuth 2.0 for Mobile & Desktop Apps:
- From the GCP Console, in the target GCP Project, go to APIs & Services > Credentials.
- Do Create credentials > OAuth client ID.
- Select Application type, either “Desktop app” (the most common type used with gargle) or “Web application” (useful for the pseudo-OOB flow).
- You can capture the client ID and secret via clipboard at this point or, as we recommend, download the full information as JSON. We recommend using the JSON, as this conveys the client type (desktop vs. web) and any redirect URIs (important for the web type).
- At any time, you can navigate to a particular client ID and click “Download JSON”.
Two ways to package this info for use with gargle:
gargle::gargle_oauth_client_from_json(). This is the preferred workflow, because the JSON conveys the client type (desktop vs. web) and any redirect URIs (important for the web type).
- Provide the path to the downloaded JSON file.
In both cases, I suggest you devise a nickname for each OAuth
credential and use it as the credential’s name in GCP Console and as the
name argument to
Package maintainers might want to build this client in as a fallback, possibly taking some measures to obfuscate the client ID and secret and limit its use to your package.
- Note that three-legged OAuth always requires the involvement of a user, so the word “secret” here can be somewhat confusing. It is not a secret in the same sense as a password or token. But you probably still want to store it in an opaque way, so that someone else cannot easily “borrow” it and present an OAuth consent screen that impersonates your package.
Package users could register their own client for use with a wrapper
package. For example, in googledrive, one would use
googledrive::drive_auth_configure() to do this:
For a long time, the recommended way to make authorized requests to an API in a non-interactive context was to use a service account token. As of April 2021, an alternative exists – workload identity federation – which is available to applications running on specific non-Google Cloud platforms, such as AWS. If you can use workload identity federation, you probably should (see the next section). But for those who can’t, here we outline the use of a conventional service account.
An official overview of service accounts is given in this official documentation by Google. But note that it’s not necessary to understand all of that in order to use a service account token.
- From the GCP Console, in the target GCP Project, go to IAM & Admin > Service accounts.
- Give it a decent name and description.
- For example, the service account used to create the googledrive docs has name “googledrive-docs” and description “Used when generating googledrive documentation”.
- Service account permissions. Whether you need to do anything here
depends on the API(s) you are targetting. You can also modify roles
later and iteratively sort this out.
- For example, the service account used to create the googledrive docs does not have any explicit roles.
- The service account used to test bigrquery has roles BigQuery Admin and Storage Admin.
- Grant users access to this service account? So far, I have not done this, so feel free to do nothing here. Or if you know this is useful to you, then by all means do so.
- Do Create key and download as JSON. This file is what we
mean when we talk about a “service account token” in the documentation
of gargle and packages that use gargle.
pathto this file.
- Appreciate that this JSON file holds sensitive information. Treat it like a username & password combo! This file holds credentials that potentially have a lot of power and that don’t expire.
- Consider storing this file in such a way that it will be
automatically discovered by the Application Default Credentials search
- You will notice the downloaded JSON file has an awful name, so sometimes I create a symlink that uses the service account’s name, to make it easier to tell what this file is.
- Remember to grant this service account the necessary permissions on any resources you plan to access, e.g., read or write permission on a specific Google Sheet. The service account has no formal relationship to you as a Google user and won’t automatically inherit permissions.
Authors of wrapper packages can use the symmetric encryption strategy described in Managing tokens securely to use this token on remote servers, such as continuous integration services like GitHub Actions.
You could provide the token’s filepath to a wrapper package’s main auth function, e.g.:
# googledrive drive_auth(path = "/path/to/your/service-account-token.json")
Alternatively, you could put the token somewhere (or store its location in an environment variable) so that it is auto-discovered by the Application Default Credentials search strategy.
Workload identity federation is a new (as of April 2021) keyless authentication mechanism that allows applications running on a non-Google Cloud platform, such as AWS, to access Google Cloud resources without using a conventional service account token. This eliminates the dilemma of how to safely manage service account credential files.
Unlike service accounts, the configuration file for workload identity federation contains no secrets. Instead, it holds non-sensitive metadata. The external application obtains the needed sensitive data “on-the-fly” from the running instance. The combined data is then used for a token exchange that ultimately yields a short-lived GCP access token. This access token allows the external application to impersonate a service account and inherit the permissions of the service account to access GCP resources.
So what’s not to love? Well, first, this auth flow is only available if your code is running on AWS, Azure, or another platform that supports the OpenID Connect protocol. Second, there’s a non-trivial amount of pre-configuration necessary on both ends. But once that is done, you can download a configuration file that makes auth work automagically with gargle.
This feature is still experimental in gargle and currently
only supports AWS. For more, see the documentation for
credentials_external_account(). Like conventional service
account tokens, workload identity federation is a great fit for the
Application Default Credentials strategy for discovering credentials.
credentials_app_default() for more about that.
These two links provide, respectively, a high-level overview and step-by-step instructions for this flow:
Learn more in Google’s documentation:
- Credentials, access, security, and identity:
- Using OAuth 2.0 for Installed Applications