![]() ![]() The program we want to write is simple: it needs to read rows from a CSVįile, extract the fields of interest, and insert rows into the sqlite knowledge of how to use raw SQL – we won’t use Gorm for this taskĪnd we’ll do it in less than 75 lines of code.a command that loads data from the Goodreads CSV export into.It just needs a backend administrativeĬommand to perform the import as a one-off process. I only need to do this once, so I don’t need to mess around I’ve got a CSV dump of my Goodreads books, and I want to import those intoĪklatan’s db. ![]() Administrative process like backups,ĭatabase migrations, or – the thing we’re going to do today – bulk data Sometimes there are operations that need to be done on the app that don’t Python’s great support for sqlite will make you love it in no time.This is the eighth in a series of articlesĪbout writing a small reading list app in Go for personal use. It’s a great database when you’d like relational database query functionality without the overhead of Postgres. Sqlite databases are great for local experimentation and are used extensively on mobile phones. Python’s build in sqlite library coupled with Pandas DataFrames makes it easy to load CSV data into sqlite databases. pd.read_sql('''SELECT * FROM users u LEFT JOIN orders o ON u.user_id = o.user_id''', conn) Next steps You can also read the SQL query directly into a Pandas DataFrame. Here’s the array that’s returned: [(1, 'pokerkid', 1, 1, 'speaker'), Join the users and orders tables on the user_id value and print the results: c.execute('''SELECT * FROM users u LEFT JOIN orders o ON u.user_id = o.user_id''') Orders.to_sql('orders', conn, if_exists='append', index = False) # write to sqlite table Fetch results of database join Orders = pd.read_csv('orders.csv') # load to DataFrame c.execute('''CREATE TABLE orders (order_id int, user_id int, item_name text)''') Suppose you have the following orders.csv file: order_id,user_id,item_nameĬreate a table and then load the orders data into the database. Cursors can be thought of as iterators in the database world. The fetchall() method returns an array of tuples.Ĭ.execute() returns a sqlite3.Cursor object. Fetch values from sqlite tableįetch all the rows from the users table: c.execute('''SELECT * FROM users''').fetchall() # The to_sql method makes it easy to write DataFrames to databases. Users.to_sql('users', conn, if_exists='append', index = False) Pandas makes it easy to load this CSV data into a sqlite table: import pandas as pd Suppose you have the following users.csv file: user_id,username ![]() c.execute('''CREATE TABLE users (user_id int, username text)''') Load CSV file into sqlite table import sqlite3Įxecute a query that’ll create a users table with user_id and username columns. You can create the file with touch my_data.db or with this equivalent Python code: from pathlib import PathĪ zero byte text file is a great starting point for a lightweight database! Creating sqlite tableĬreate a database connection and cursor to execute queries. Sqlite is a lightweight database that can be started as an empty text file. Python is perfect language for this task because it has great libraries for sqlite and CSV DataFrames. This blog post demonstrates how to build a sqlite database from CSV files. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |