The idea
Let’s demonstrate the basic functionality on a useful idea. I have a folder/label in Gmail with a lot of emails. I want to list all email addresses - all senders.
Everything I need to do is a small console app to go through all emails with the given label and extracting the From header. If it contains the email address in format Name <[email protected]>, extract only the email address.
Let’s dive into it!
Get credentials
Before we start, you need to create a new project in Google Cloud Console.
Under the new project, navigate to API & Services and enable Gmail API in Library.
In Credentials, click the + CREATE CREDENTIALS, select OAuth client ID and setup it like this:
Click Save and download the credentials for your newly created ID:
Save the downloaded file as credentials.json.
Kotlin project
Create a new Kotlin project with Gradle and add Google’s dependencies:
dependencies {
implementation "org.jetbrains.kotlin:kotlin-stdlib"
implementation 'com.google.api-client:google-api-client:1.23.0'
implementation 'com.google.oauth-client:google-oauth-client-jetty:1.23.0'
implementation 'com.google.apis:google-api-services-gmail:v1-rev83-1.23.0'
implementation 'org.jetbrains.kotlinx:kotlinx-coroutines-core:1.4.0-M1'
}
Authorization scopes
As we only want to go through labels, messages (we need just headers - metadata), we need a small and safe set of scopes:
private val SCOPES = setOf(
GmailScopes.GMAIL_LABELS,
GmailScopes.GMAIL_READONLY,
GmailScopes.GMAIL_METADATA
)
Beware that if you provide the GmailScopes.GMAIL_METADATA, you are not able to access the whole message. You have to omit it if you want to get the message body.
Authorize with Gmail
Fortunately, Google libraries come with everything we may need including the server for receiving the authorization request. The whole implementation is as simple as:
private fun getCredentials(httpTransport: NetHttpTransport): Credential? {
val inputStream = File("credentials.json").inputStream()
val clientSecrets = GoogleClientSecrets.load(JSON_FACTORY, InputStreamReader(inputStream))
val flow = GoogleAuthorizationCodeFlow.Builder(httpTransport, JSON_FACTORY, clientSecrets, SCOPES)
.setDataStoreFactory(FileDataStoreFactory(File(TOKENS_DIRECTORY_PATH)))
.setAccessType("offline")
.build()
val receiver = LocalServerReceiver.Builder().setPort(8888).build()
return AuthorizationCodeInstalledApp(flow, receiver).authorize("user")
}
This code above outputs the request for authorization to the console:
Please open the following address in your browser:
https://accounts.google.com/o/oauth2/auth?access_type=offline&client_id=...
Click the link, authorize the app, and the authorization token is received and stored in TOKENS_DIRECTORY_PATH. You only need to do this for the first time. Next time, the stored token is used.
Build client & get labels
We can now use the getCredentials() function above to build an authorized client, list all labels, and find the required one identified by labelName.
// Build a new authorized API client service.
val httpTransport = GoogleNetHttpTransport.newTrustedTransport()
val service = Gmail.Builder(httpTransport, JSON_FACTORY, getCredentials(httpTransport))
.setApplicationName(APPLICATION_NAME)
.build()
// Find the requested label
val user = "me"
val labelList = service.users().labels().list(user).execute()
val label = labelList.labels
.find { it.name == labelName } ?: error("Label `$labelName` is unknown.")
List all emails
For listing all email messages, let’s use a few of Kotlin’s goodies - tailrec extension function with lambda as the last parameter.
We need to invoke the list request repeatedly until the nextPageToken is null, and doing so with tailrec is safer.
For each message, we invoke the process lambda to perform an actual operation.
private tailrec fun Gmail.processMessages(
user: String,
label: Label,
nextPageToken: String? = null,
process: (Message) -> Unit
) {
val messages = users().messages().list(user).apply {
labelIds = listOf(label.id)
pageToken = nextPageToken
includeSpamTrash = true
}.execute()
messages.messages.forEach { message ->
process(message)
}
if (messages.nextPageToken != null) {
processMessages(user, label, messages.nextPageToken, process)
}
}
Process message
The code for listing emails above returns only id and threadId for each of the messages, so we need to fetch message details, extract From header, and eventually process it.
To speed up the process, let’s use Kotlin’s coroutines to perform the message fetching in parallel. First, introduce a custom dispatcher, so we can limit the number of threads.
private val MAX_FETCH_THREADS = Runtime.getRuntime().availableProcessors()
val executors = Executors.newFixedThreadPool(MAX_FETCH_THREADS)
val dispatcher = object : CoroutineDispatcher() {
override fun dispatch(context: CoroutineContext, block: Runnable) {
executors.execute(block)
}
}
For extracting the email address from the Name <[email protected]> format, the code is simple:
private fun String.parseAddress(): String {
return if (contains("<")) {
substringAfter("<").substringBefore(">")
} else {
this
}
}
Now, we can put things together. Of course, you should introduce some logic for catching exceptions, etc.
private fun Gmail.processFroms(
user: String,
label: Label,
process: (String) -> Unit
) {
runBlocking(dispatcher) {
processMessages(user, label) { m ->
launch {
val message = users().messages().get(user, m.id).apply { format = "METADATA" }.execute()
message.payload.headers.find { it.name == "From" }?.let { from ->
process(from.value.parseAddress())
}
}
}
}
}
Result
With all the code above, we can get unique list of all senders like this:
val senders = mutableSetOf<String>()
service.processFroms(user, label) {
senders += it
}
senders.forEach(::println)
Source code
The complete source code is available on Github.
This is a companion discussion topic for the original entry at https://localazy.com/blog/kotlin-gmail-api-listing-emails