Airlift

An automated method to upload & merge *.csv or *.json data files with attachments to Airtable database.

Core Features

Automated uploading of .csv or .json data to Airtable
Ability to update and auto-create new entries for single select field and multiple select field
No subscription of third party platform required
Ability to upload attachments via build-in Dropbox client
Ability to upload attachments to multiple attachment columns
Uses long-lived refresh token for Dropbox client

Background

Airlift draws inspiration from our very own CSV2Notion Neo, which was inspired by Airtable’s CSV import extension. And with Airlift, we have come full circle.

To send data to Airtable, users or applications need to write custom code using libraries that connect with Airtable's API. For normal end-users who aren’t developers, this can be an extremely challenging endeavor. For most users, Airtable’s CSV import extension could be sufficient. But CSV import extension is only available on paid plan.

Furthermore, many experienced Airtable users want to send and transfer data automatically without using Make or Zapier. This is where Airlift comes into play.

Airlift is free and open-source software. But you can sponsor us if you find it useful.

Installation

Pre-compiled Binary (Recommended)

Download the latest release of the latest release here.

With Homebrew (Recommended for macOS)

$ brew install TheAcharya/homebrew-tap/airlift

$ brew uninstall --cask airlift

From source

This project uses poetry for dependency management and packaging. You will have to install it first. See poetry official documentation for instructions.

$ git clone https://github.com/TheAcharya/airlift.git
$ cd airlift/
$ poetry install --no-dev
$ poetry run airlift

Guide

$ airlift --help
usage: airlift [-h] --token TOKEN --base BASE --table TABLE [OPTION]... FILE

https://github.com/TheAcharya/airlift

Upload & Merge Data with Attachments to Airtable

positional arguments:
  FILE                               CSV or JSON file to upload

general options:
  --token TOKEN                      your Airtable personal access token
  --base BASE                        your Airtable Base ID
  --table TABLE                      your Airtable Table ID
  --log FILE                         file to store program log
  --verbose                          output debug information
  --version                          show program's version number and exit
  --workers                          total number of worker threads to upload your data (default: 5)
  -h, --help                         show this help message and exit

dropbox options:
  --dropbox-token FILE               your JSON file with Dropbox API App key
  --dropbox-refresh-token            switch to change your refresh token
  --attachment-columns               specify one or more attachment columns
  --attachment-columns-map           specify how the attachment column must be mapped in Airtable

column options:
  --disable-bypass-column-creation   creates new columns that are not present in Airtable's table
  --columns-copy                     copys value of one column to multiple other columns
  --rename-key-column                rename the key column in the file to a different key column in Airtable

validation options:
  --fail-on-duplicate-csv-columns    fail if CSV has duplicate columns;
                                     otherwise last column will be used

macOS Release

Privacy & Security

For macOS, you have the option of two distinct releases: one packaged within a .zip archive and the other in a .pkg format. Navigate to the Privacy & Security settings and set your preference to App Store and identified developers.

Utilise the Airlift.pkg installer to install the command-line binary into your system. Upon completion, find the installed binary airlift located within /usr/local/bin. To uninstall, you can utalise this terminal command.

sudo rm /usr/local/bin/airlift

Prerequisite

Airtable

You must pass a single *.csv or *.json file for upload. The CSV file must contain at least 2 rows. The first row will be used as a header.

Obtain your Airtable's Personal Access Token:

Login to your Airtable account via a web browser.
Go to Personal access token, click the Create new token button to create a new personal access token.
Give your token a unique name. This name will be visible in record revision history.
Add the following scopes to grant to your token. This controls what API endpoints the token will be able to use.

Click ‘add a base’ to grant the token access to a base or workspace

You can grant access to any combination and number of bases and workspaces. You can also grant access to all workspaces and bases under your account. Keep in mind that the token will only be able to read and write data within the bases and workspaces that have been assigned to it.

Once your token is created, the token will only be shown to you once, it is encouraged that you to copy it to your clipboard and store it somewhere safe. While you will be able to manage it in Personal access token, the sensitive token itself is not stored for security purposes.

Obtain your Airtable's Base ID & Table ID:

When you have a base open in a compatible web browser, you should see a URL in the address bar that looks similar to the example below:

In between each backslash, you will find a string that identifies the base, table, and view IDs.

Base IDs begin with "app"
Table IDs begin with "tbl"
View IDs begin with "viw"

We only require Base ID and Table ID for Airlift

Dropbox

Airtable's API does not allow direct uploading of attachments. In order to upload an attachment via Airtable's API, the attachment must first exist at a publicly accessible url. To mitigate this problem, Airlift will utilise Dropbox as a temporary storage provider. Dropbox’s basic account offers 2GB of free storage. We believe it would be adequate for most operations. Once your attachments are uploaded by Airtable, you can delete the files from your Dropbox account.

Obtain your Dropbox App Key & Authorisation code (First Time Only):

Right-click and save dropbox-token.json file to your computer.
When using --dropbox-token make sure you input the full PATH of dropbox-token.json file.
Login to your Dropbox's App Console account via a web browser.
Click on ‘Create app’ button.
Choose Scoped access.
Choose Full Dropbox access.
Give your App a name. The name of the App can be unique and personal to you.
Click on ‘Create app’ button.

Go to the Permissions tab.
Set the permissions as shown on the screenshot.
Click on 'Submit' at the bottom.

Go to the Settings tab.
Copy your App key and paste into your dropbox-token.json file. Where REPLACE is your App key.

{
  "app_key": "REPLACE"
}

On first usage of Airlift, you will be promoted to visit Dropbox.
Copy and paste the full Dropbox URL into your browser.

INFO: Validation done!
INFO: All the columns are verified and present in both the file and Airtable!
1. Go to: https://www.dropbox.com/oauth2/authorize?response_type=code&client_id=6zh18qgnw37ifpp&token_access_type=offline&code_challenge=TphrwcwmRtkGawgxFvWQcROFMbjsTeba9BGv0Lgi0nw&code_challenge_method=S256
2. Click "Allow" (you might have to log in first).
3. Copy the authorization code.
Enter the authorization code here:

Click Continue.

Click Allow.

You will be presented with your authorization code. Copy your authorization code.

Enter the authorization code here:    XXXZZaa0-poAAAAAAABZNgc9CwNdyryqoRAi4fxP2aU

Paste it back into the terminal.
Airlift would update and store your Dropbox refresh token into your dropbox-token.json file.
This is a one time procress. You will not be asked again.

Do not share your access json token file with anyone.

Reset and Update your Dropbox App key (Optional):

For reasons should you decide to delete and revoke the App from Dropbox's App Console and created a new App, you would have to repeate the above mentioned steps.
For easy updating of your your Dropbox App key, right-click and save reset-create-dropbox-app-key.sh file to the same location of your dropbox-token.json file.
To execute your script, open Terminal, type sh and drag and drop reset-create-dropbox-app-key.sh file into the terminal.
You will promted to enter your Dropbox App key.
The script would overwrite dropbox-token.json file with your new Dropbox App key.

Creating your Dropbox refresh token (Optional):

If you have created your and replaced your Dropbox App key in your dropbox-token.json file. You would need to re-create your Dropbox refresh token again.

You can utalise this this .sh shoukd you need to authorise and refresh your dropbox-token.json file.

#!/bin/sh

TOOL_PATH="/Users/xxx/Desktop/Airlift/airlift"
DROPBOX_TOKEN="/Users/xxx/Desktop/Airlift/dropbox-token.json"
UPLOAD_LOG="/Users/xxx/Desktop/Airlift/log.txt"

$TOOL_PATH --dropbox-token $DROPBOX_TOKEN --dropbox-refresh-token --log $UPLOAD_LOG

Column Types

By default, Airlift will try to guess column types based on their content.

Some column types do not support assigning value to them because the database generates their content automatically.

Support Table

Column Type Name	Supported Values	Multiple Values (Comma Separated)
Attachment	string	❌
Autonumber	numerical	❌
Barcode	string	❌
Button	`---`	`---`
Checkbox	`true`, `false`	❌
Count	numerical	❌
Created time	`---`	`---`
Created by	`---`	`---`
Currency	numerical	❌
Date & Time	string	❌
Duration	string	❌
Email	string	❌
Formula	`---`	`---`
Last modified by	`---`	`---`
Last modified time	`---`	`---`
Linked record	string	✅
Long text	string	❌
Lookup	`---`	`---`
Multiple select	string	✅
Number	numerical	❌
Percent	numerical	❌
Phone number	string	❌
Rating	numerical	❌
Rollup	`---`	`---`
Single line text	string	❌
Single select	string	❌
URL	string	❌
User	`---`	`---`

Examples

Basic creation of folders and .sh file

For ease of use, usage and creation of .sh files is recommended.

Create a folder called Airlift on your Desktop.
Place the latest pre-compiled binary with the folder.
Right-click and save dropbox-token.json file and save it within your Airlift.
Within that folder, create a folder, Data.
Data is where you place your .csv or .json and attachments files.
Create a file using any text editor. Name the script file with extension .sh
Copy and paste this syntax into the file, where xxx is the name of of your user directory and zzz is the name of your .csv or .json file.
REPLACE the text with your relevent tokens and IDs.
Save the script file as myscript.sh within your Airlift folder.
To give execute permission to your script, open Terminal, chmod +x /Users/xxx/Desktop/Airlift/myscript.sh
To execute your script, open Terminal, sh /Users/xxx/Desktop/Airlift/myscript.sh
To obtain your Dropbox App Key and Authorization code, please read the Dropbox section.
You can create and save multiple .sh files for different modes and configurations.

Uploading Data without Attachments

#!/bin/sh

TOOL_PATH="/Users/xxx/Desktop/Airlift/airlift"
AIRTABLE_TOKEN="REPLACE"
AIRTABLE_BASE="REPLACE"
AIRTABLE_TABLE="REPLACE"
UPLOAD_PAYLOAD="/Users/xxx/Desktop/Airlift/Data/zzz.csv"
UPLOAD_LOG="/Users/xxx/Desktop/Airlift/log.txt"

$TOOL_PATH --token $AIRTABLE_TOKEN --base $AIRTABLE_BASE --table $AIRTABLE_TABLE --log $UPLOAD_LOG "$UPLOAD_PAYLOAD"

Example CSV

Cat ID,Animal Name,Location
001,Lion,Namibia
002,Tiger,India
003,Panther,India
004,Snow Leopard,Nepal
005,Cheetah,South Africa
006,Puma,South America
007,Jaguar,Brazil

Uploading Data without Attachments (Copy Columns)

#!/bin/sh

TOOL_PATH="/Users/xxx/Desktop/Airlift/airlift"
AIRTABLE_TOKEN="REPLACE"
AIRTABLE_BASE="REPLACE"
AIRTABLE_TABLE="REPLACE"
UPLOAD_PAYLOAD="/Users/xxx/Desktop/Airlift/Data/zzz.csv"
UPLOAD_LOG="/Users/xxx/Desktop/Airlift/log.txt"

$TOOL_PATH --token $AIRTABLE_TOKEN --base $AIRTABLE_BASE --table $AIRTABLE_TABLE --columns-copy "Location" "Location-01" "Location-02" "Location-03" --log $UPLOAD_LOG "$UPLOAD_PAYLOAD"

Example CSV

Cat ID,Animal Name,Location
001,Lion,Namibia
002,Tiger,India
003,Panther,India
004,Snow Leopard,Nepal
005,Cheetah,South Africa
006,Puma,South America
007,Jaguar,Brazil

Uploading Data with Attachments

#!/bin/sh

TOOL_PATH="/Users/xxx/Desktop/Airlift/airlift"
AIRTABLE_TOKEN="REPLACE"
AIRTABLE_BASE="REPLACE"
AIRTABLE_TABLE="REPLACE"
DROPBOX_TOKEN="/Users/xxx/Desktop/Airlift/dropbox-token.json"
UPLOAD_PAYLOAD="/Users/xxx/Desktop/Airlift/Data/zzz.csv"
UPLOAD_LOG="/Users/xxx/Desktop/Airlift/log.txt"

$TOOL_PATH --token $AIRTABLE_TOKEN --base $AIRTABLE_BASE --table $AIRTABLE_TABLE --dropbox-token $DROPBOX_TOKEN --attachment-columns "Image Filename" --log $UPLOAD_LOG "$UPLOAD_PAYLOAD"

Example CSV

Cat ID,Animal Name,Location,Image Filename
001,Lion,Namibia,lion.jpg
002,Tiger,India,tiger.jpg
003,Panther,India,panther.jpg
004,Snow Leopard,Nepal,snow_leopard.jpg
005,Cheetah,South Africa,cheetah.jpg
006,Puma,South America,puma.jpg
007,Jaguar,Brazil,jaguar.jpg

Example Folder Structure

Desktop/
├─ Airlift/
│  ├─ Data/
│  │  ├─ zzz.csv
│  │  ├─ cheetah.jpg
│  │  ├─ jaguar.jpg
│  │  ├─ lion.jpg
│  │  ├─ panther.jpg
│  │  ├─ puma.jpg
│  │  ├─ snow_leopard.jpg
│  │  ├─ tiger.jpg
│  ├─ myscript.sh
│  ├─ dropbox-token.json
│  ├─ airlift [Binary Executable File]

Uploading Data with Multiple Attachments Column

#!/bin/sh

TOOL_PATH="/Users/xxx/Desktop/Airlift/airlift"
AIRTABLE_TOKEN="REPLACE"
AIRTABLE_BASE="REPLACE"
AIRTABLE_TABLE="REPLACE"
DROPBOX_TOKEN="/Users/xxx/Desktop/Airlift/dropbox-token.json"
UPLOAD_PAYLOAD="/Users/xxx/Desktop/Airlift/Data/zzz.csv"
UPLOAD_LOG="/Users/xxx/Desktop/Airlift/log.txt"

$TOOL_PATH --token $AIRTABLE_TOKEN --base $AIRTABLE_BASE --table $AIRTABLE_TABLE --dropbox-token $DROPBOX_TOKEN --attachment-columns "Colour Image" "Black & White Image" --log $UPLOAD_LOG "$UPLOAD_PAYLOAD"

Example CSV

Cat ID,Animal Name,Location,Colour Image,Black & White Image
001,Lion,Namibia,Colour/lion.jpg,Black & White/lion.jpg
002,Tiger,India,Colour/tiger.jpg,Black & White/tiger.jpg
003,Panther,India,Colour/panther.jpg,Black & White/panther.jpg
004,Snow Leopard,Nepal,Colour/snow_leopard.jpg,Black & White/snow_leopard.jpg
005,Cheetah,South Africa,Colour/cheetah.jpg,Black & White/cheetah.jpg
006,Puma,South America,Colour/puma.jpg,Black & White/puma.jpg
007,Jaguar,Brazil,Colour/jaguar.jpg,Black & White/jaguar.jpg

Example Folder Structure

Desktop/
├─ Airlift/
│  ├─ Data/
│  │  ├─ zzz.csv
│  │  ├─ Black & White/
│  │  │  ├─ cheetah.jpg
│  │  │  ├─ jaguar.jpg
│  │  │  ├─ lion.jpg
│  │  │  ├─ panther.jpg
│  │  │  ├─ puma.jpg
│  │  │  ├─ snow_leopard.jpg
│  │  │  ├─ tiger.jpg
│  │  ├─ Colour/
│  │  │  ├─ cheetah.jpg
│  │  │  ├─ jaguar.jpg
│  │  │  ├─ lion.jpg
│  │  │  ├─ panther.jpg
│  │  │  ├─ puma.jpg
│  │  │  ├─ snow_leopard.jpg
│  │  │  ├─ tiger.jpg
│  ├─ myscript.sh
│  ├─ dropbox-token.json
│  ├─ airlift [Binary Executable File]

Uploading Data with Attachments using Mapping

#!/bin/sh

TOOL_PATH="/Users/xxx/Desktop/Airlift/airlift"
AIRTABLE_TOKEN="REPLACE"
AIRTABLE_BASE="REPLACE"
AIRTABLE_TABLE="REPLACE"
DROPBOX_TOKEN="/Users/xxx/Desktop/Airlift/dropbox-token.json"
UPLOAD_PAYLOAD="/Users/xxx/Desktop/Airlift/Data/zzz.csv"
UPLOAD_LOG="/Users/xxx/Desktop/Airlift/log.txt"

$TOOL_PATH --token $AIRTABLE_TOKEN --base $AIRTABLE_BASE --table $AIRTABLE_TABLE --dropbox-token $DROPBOX_TOKEN --attachment-columns-map "Image Filename" "Attachments" --log $UPLOAD_LOG "$UPLOAD_PAYLOAD"

Example CSV

Cat ID,Animal Name,Location,Image Filename
001,Lion,Namibia,lion.jpg
002,Tiger,India,tiger.jpg
003,Panther,India,panther.jpg
004,Snow Leopard,Nepal,snow_leopard.jpg
005,Cheetah,South Africa,cheetah.jpg
006,Puma,South America,puma.jpg
007,Jaguar,Brazil,jaguar.jpg

Example Folder Structure

Desktop/
├─ Airlift/
│  ├─ Data/
│  │  ├─ zzz.csv
│  │  ├─ cheetah.jpg
│  │  ├─ jaguar.jpg
│  │  ├─ lion.jpg
│  │  ├─ panther.jpg
│  │  ├─ puma.jpg
│  │  ├─ snow_leopard.jpg
│  │  ├─ tiger.jpg
│  ├─ myscript.sh
│  ├─ dropbox-token.json
│  ├─ airlift [Binary Executable File]

Uploading Data with Attachments (Rename Key Column + Mapping)

#!/bin/sh

TOOL_PATH="/Users/xxx/Desktop/Airlift/airlift"
AIRTABLE_TOKEN="REPLACE"
AIRTABLE_BASE="REPLACE"
AIRTABLE_TABLE="REPLACE"
DROPBOX_TOKEN="/Users/xxx/Desktop/Airlift/dropbox-token.json"
UPLOAD_PAYLOAD="/Users/xxx/Desktop/Airlift/Data/zzz.csv"
UPLOAD_LOG="/Users/xxx/Desktop/Airlift/log.txt"

$TOOL_PATH --token $AIRTABLE_TOKEN --base $AIRTABLE_BASE --table $AIRTABLE_TABLE --dropbox-token $DROPBOX_TOKEN --attachment-columns-map "Image Filename" "Attachments" --rename-key-column "Cat ID" "Animal ID" --log $UPLOAD_LOG --verbose "$UPLOAD_PAYLOAD"

Example CSV

Cat ID,Animal Name,Location,Image Filename
001,Lion,Namibia,lion.jpg
002,Tiger,India,tiger.jpg
003,Panther,India,panther.jpg
004,Snow Leopard,Nepal,snow_leopard.jpg
005,Cheetah,South Africa,cheetah.jpg
006,Puma,South America,puma.jpg
007,Jaguar,Brazil,jaguar.jpg

Example Folder Structure

Desktop/
├─ Airlift/
│  ├─ Data/
│  │  ├─ zzz.csv
│  │  ├─ cheetah.jpg
│  │  ├─ jaguar.jpg
│  │  ├─ lion.jpg
│  │  ├─ panther.jpg
│  │  ├─ puma.jpg
│  │  ├─ snow_leopard.jpg
│  │  ├─ tiger.jpg
│  ├─ myscript.sh
│  ├─ dropbox-token.json
│  ├─ airlift [Binary Executable File]

Future Roadmap

Ability to create new Airtable Bases
Ability to Update & Add New Unique Records
Robust Validation Options
And many more...

Use Cases

Get the most out of your Airtable Free Account (CSV import extension is only available on paid plans)
No reliance on third-party automation platforms to sync your local .csv or .json data files to Airtable
Run data synchronisation locally
On macOS: Utilise .sh scripts, Apple Shortcuts or Automator to synchronise your local data to Airtable
On Windows: Utilise Powershell or .bat files to synchronise your local data to Airtable
On Linux: Utilise .sh scripts or to synchronise your local data to Airtable

Utilised By

Marker Data

Marker Data's Notion Panel

Credits

Original Idea and Workflow Architecture by Vigneswaran Rajkumar

Maintained by Arjun Prakash (1.0.0 ...)

Icon Design by Bor Jen Goh

License

Licensed under the MIT license. See LICENSE for details.

Reporting Bugs

For bug reports, feature requests and other suggestions you can create a new issue to discuss.

Name		Name	Last commit message	Last commit date
Latest commit History 535 Commits
.github		.github
airlift		airlift
assets		assets
bin		bin
.gitignore		.gitignore
CHANGELOG.md		CHANGELOG.md
CODE_OF_CONDUCT.md		CODE_OF_CONDUCT.md
CONTRIBUTING.md		CONTRIBUTING.md
LICENSE		LICENSE
README.md		README.md
SECURITY.md		SECURITY.md
airlift.spec		airlift.spec
poetry.lock		poetry.lock
pyproject.toml		pyproject.toml

License

TheAcharya/Airlift

Folders and files

Latest commit

History

Repository files navigation

Airlift

Core Features

Table of contents

Background

Installation

Pre-compiled Binary (Recommended)

With Homebrew (Recommended for macOS)

From source

Guide

macOS Release

Prerequisite

Airtable

Dropbox

Column Types

Examples

Future Roadmap

Use Cases

Utilised By

Marker Data

Credits

License

Reporting Bugs

About

Topics

Resources

License

Code of conduct

Security policy

Stars

Watchers

Forks

Sponsor this project

Languages