Anvil apps have a full Python instance server-side, which makes it simple to import data from CSVs and Excel files into your Data Tables.
The PBI queries bring the CDS data together with the Oracle data to output the finalized CSV files. Lastly, I want an Excel workbook that has all the data appended back together for use as a data source within PowerApps. This is why I’m trying to get the Python script to export to xlsx. I usually create dataframes by reading data from csv, excel and sql. You can find some examples here: python reads data from SQL directly; python executes sql query. Python Pandas read csv common problems. Python Pandas read excel files (xlsx) But sometimes, you may want to create a small dataframe in the script. So here are some examples.
There are two ways to do this:
- Using a FileLoader Component in a running Anvil app
- Accessing files locally using the Anvil Uplink
We’ll cover both methods, and you can download our sample CSV and Excel files here:
File upload in a running app
First, we’ll import data to Anvil’s Data Tables using a FileLoader component in a running Anvil app.
Create a new app, add the Data Tables service, and create a table.
Ensure that ‘Auto-create missing columns when adding rows’ in the top right hand corner is checked:
Add a Server Module, and select the ‘Full Python 3’ runtime in the dropdown in top right:
pandas
library which requires the Full Python server instance, available on any of our paid plans.Add these lines to your Server Module:
your_table_name_here
to the name of the Data Table you just created!Go back to ‘Form1’, and drop a FileLoader component into your UI.
The FileLoader’s change
event is raised when a user selects a file. Its file
argument is a Media object containing the chosen file.
Double click the FileLoader in your UI to create the file_loader_1_change
event, and edit the function to look like this:
Run your app, and upload a CSV or excel file into the FileLoader component.
Stop your app, navigate to your Data Tables, and you’ll see your data has been imported.
Uplink script
If you’d rather access your files locally with Python, rather than uploading them in a running Anvil app, you can use the Uplink to do this.
Create a new app, add the Data Tables service, and create a table.
Ensure that ‘Auto-create missing columns when adding rows’ in the top right hand corner is checked:
We’ll use the pandas
and xlrd
libraries, so install these:
Import from CSV
This Python script will import data from a CSV file to your Data Table:
your_table_name_here
to the name of the Data Table you just created!Import from Excel
This Python script will import data from an Excel file to your Data Table:
your_table_name_here
to the name of the Data Table you just created!Next, install the Anvil Uplink library:
Enable the Uplink in your app, and then paste the connection code into your script:
Run your scripts
Run your script, calling your functions and passing in the CSV or Excel files you want to import.
Stop your app, navigate to your Data Tables, and you’ll see your data has been imported.
Example script
Here’s an example script that uploads both colours.csv
and colours.xlsx
to an Anvil Data Table:
Introduction
I have written severaltimes about the usefulness of pandas as a data manipulation/wranglingtool and how it can be used to efficiently move data to and from Excel.There are cases, however, where you need an interactive environment for data analysisand trying to pull that together in pure python, in a user-friendly manner would be difficult.This article will discuss how to use xlwings to tie Excel, Python and pandas togetherto build a data analysis tool that pulls information from an external database,manipulates it and presents it to the user in a familiar spreadsheet format.
A Quick Excel Automation Intro
Excel supports several automation options using VBA. User Defined Functions (UDF)are relatively simple in that they take inputs and returns a single value.The more powerful option is a macro (or procedure) that can automate just about anythingExcel can do.
Despite the fact that UDF’s and macros are powerful, they are still written in VBA and thereare times when it would be useful to bring the power of python to our Excel-basedsolution. That’s where xlwings comes into play. At the simplest level, xlwings allowsus to glue python and Excel together in two main ways:
- Control Excel from python
- Call custom python code from within Excel
This article will focus on building an Excel worksheet that calls your custom python code.
The Problem
For this example, we are going to develop a simple modeling application thatwill allow someone to enter an account number and date range then return somesummarized sales information that has been transformed via pandas. The solution issimple but shows the power of this combination and how easily you could performmore complex data analysis.
Here’s a diagram of what we are trying to do:
The example shown below could easily be expanded to query multiple databases or interactwith any kind of file that python can read (CSV, Excel, json, etc.)
Setting Up The Environment
For the purposes of this article, I will assume you are running the applicationon a Windows-based system. I highly recommend you use anaconda (or miniconda)as your distro of choice.
The first thing we need to do is install xlwings (assuming python+pandas are already installed):
There is a nice xlwings helper function called quickstart
which will create a sample Excel fileand stub python file for you.
If you look in the newly created pbp_proj directory, you’ll see two files:
The python file is empty and the Excel file looks empty but there has been somebehind the scenes work done to make the excel to python interface easier for you.
To see what is put into the Excel file, open your newly created file in Excel andgo into Developer -> Visual Basic and you should see something like this:
You will notice that there are two modules - xlwings
and Module1
. The xlwingsmodule includes all the VBA code to make your custom code work. For the mostpart you should leave that alone. However, if you have issues with your configuration(like you can’t find python) then you can update the config information in this section.
The Module1
will have some default code that looks like this:
We will modify that in a moment to call our custom code. First, I want to create theExcel input fields.
For this application, we are going to allow the user to enter an accountnumber, start date and end date and will manipulate the sales date based on these inputs.
Here is the simple spreadsheet:
I have only made some minor formatting changes, there are no formulas in the cells.Be sure to save the changes to the Excel file.
For the next step, I’m going to create a short python function that illustrateshow to read data from Excel and write it back. I will be saving this in theempty file called pbp_proj.py
The program is simple and not very useful at this point. I think it is easierto develop a skeleton program in order to make sure all the “plumbing” is in place.The key thing to remember is that the file is called pbp_proj.py
and thefunction is called summarize_sales
.
To wire this all together, we need to define an Excel procedure to run our code:
The code is really concise just import the module and execute the function:
The final piece is to add a button to our sheet and assign it to theprocedure/macro RetrieveSales
.
Once you have that in place, you should be able to press the button and see something like this:
The basic process is in place. We can read from Excel into a python program and usethat to output data back into Excel. Now, let’s make this a little more useful.
Reading From a Database
For this example, I’m going to use sqlalchemy to query a small sqlite db and read thatquery directly into a pandas dataframe. The nice thing about this approach is that ifyou decide that you want to query another database, you can just change the slqlalchemyengine and keep the rest of your code the same. For reference, the xlwings siteshows another example that should be helpful as a further reference.
Before proceeding with the code, make sure sqlalchemy is installed:
Convert Csv File To Excel Python
Here is how to connect to the sqlite engine by using the full path to the database:
Now that we have the engine, we can construct and execute the query and read theresults into a dataframe:
Once we have the data in the sales_data
dataframe, we can do anything we want with it.For the sake of simplicity, I will do a simple groupby
then a sum
of the total spend:
Fortunately xlwings “understands” a pandas dataframe so placing the value back inthe Excel sheet is straightforward:
That completes the round trip of data from Excel -> Python -> Excel.
Full Program
Here is the fully functioning code included in pbp_proj.py
Python Csv To Excel File
Here is a sample result:
All of the data, including the sqlite db is in my github repo.
Summary
Csv File To Excel Python
xlwings provides a useful capability to interact seamlessly with Excel from python.By using this code, you can easily build interactive tools for yourself orfor less technical users that pull data from multiple sources and analyze it inthe very familiar Excel environment. Once the structure is set up, it is really usefulto put all your complex logic and data analysis in the python file and harness allthe tools available in the python ecosystem. I hope that once you start to playwith this, you will find lots of opportunities to use this approach to bring pythonsolutions to some of your less technical users who are stuck using Excel as theironly tool for data analysis.