Django Import Export: Goodbye Manual Data Entry

Sumire Banh -

Suppose you're a Services Project Assistant at a company with highly trusted users. Each month, numerous projects require you to create many work packages in the administrative system to manage these projects. What's the situation? Manually creating 100 work packages per project, with two cups of coffee a day? It seems like it's time to 'knock knock' on the developer team's door and ask them for help, perhaps with a script or something similar. If your system is built on the Django framework, your developer might tell you to wait just 10 minutes to solve what would otherwise be a day's problem for you.

That's the reason for this blog's existence. I am that developer, and I will now guide you (hopefully another developer, not the admin ^^) on how to assist your admin.

Prerequisites

Everything starts with an installation command. This is the step where you can grab a cup of coffee and wait until it's completed.

pip install django-import-export

After successful completion, locate the settings.py file and add import_export to INSTALLED_APP:

# settings.py
INSTALLED_APPS = (
    ...
    'import_export',
)

Simple, right? Now let's move on to coding, it's just as straightforward.

Implementation Journey

Here is a snapshot of my models.py file. If you're unsure why we have this file, check this:: Django Models.

class Project(models.Model):
    id = models.BigAutoField(primary_key=True)
    project_name = models.CharField(unique=True, max_length=255)
    ref_code = models.CharField(max_length=255, blank=True, null=True)
    status = models.CharField(max_length=50, choices=Status.choices, default=Status.ACTIVE)
    
    class Meta:
        db_table = "projects"
        
    def  __str__(self):
        return self.project_name
        
        
class WorkPackage(models.Model):
    id = models.BigAutoField(primary_key=True)
    project = models.ForeignKey(Project, on_delete=models.CASCADE, blank=True, null=True)
    work_package_name = models.CharField(max_length=255)
    status = models.CharField(max_length=50, choices=Status.choices, default=Status.ACTIVE)
    
    class Meta:
        db_table = "work_packages"
    
    def __str__(self):
        return self.work_package_name

Returning to the admin's problem, suppose he or she has a CSV file structured like this:

Take a look at this file. We need to import data for Work Package model instance which has a foreign key relationship to Project model, use project_name as the lookup field. Thankfully, Django Import Export can  can handle the lookup and linking to the related model.

Let's create a resources.py file:

from import_export import resources, fields
from .models import Project, WorkPackage
from import_export.widgets import ForeignKeyWidget

class WorkPackageResource(resources.ModelResource):
    project = fields.Field(
       column_name = 'Project Name'  # The column name (i.e., row header) in the CSV file
       attribute = 'project'  # This field represents the foreign key field
       widget = ForeignKeyWidget(Project, 'project_name')  # Reference the Project model, using project_name as the lookup field
   )
   
   work_package_name = fields.Field(
       column_name='Work Packages',
       attribute='work_package_name'
   )
   
   class Meta:
       model = WorkPackage
       fields = ('id', 'project', 'work_package_name')
       import_id_fields = ('project', 'work_package_name')
       skip_unchanged = True
       report_skipped = True
       use_transactions = True

Now, let's move to the admin.py file and intergrate import-export:

from .models import Project, WorkPackage
from .resources import WorkPackageResource
from import_export.admin import ImportExportModelAdmin

class WorkPackageAdmin(ImportExportModelAdmin, admin.ModelAdmin):
    resource_classes = [WorkPackageResource]

We're done. Let's take a look at our results:

Tada! The Import and Export buttons now appear on the Work Package UI. Click Import to see what's next:

Try uploading the CSV file to test:

Oh no, we received a lot of warning errors, all with the same message: Project matching query does not exist. This is because Django Import Export cannot find any project with the name Django Import Export - Help me. What should we do next? Should we DM the admin and ask them to create it first? It would be manageable if there was only one missing project, but what if there are 50? The task would be pending until they are all created. Let's add some code to handle this automatically.

Go back to your  resources.py file and add the widget’s clean() method:

from import_export import resources, fields
from .models import Project, WorkPackage
from import_export.widgets import ForeignKeyWidget


class ForeignKeyWidgetWithCreate(ForeignKeyWidget):
    def clean(self, value, row=None, *args, **kwargs):
        if not value:
            return None
        return Project.objects.get_or_create(**{self.field: value})[0]


class WorkPackageResource(resources.ModelResource):
    project = fields.Field(
        column_name='Project Name',
        attribute='project',
        widget=ForeignKeyWidgetWithCreate(Project, 'project_name')
    )

    work_package_name = fields.Field(
        column_name='Work Packages',
        attribute='work_package_name'
    )

    class Meta:
        model = WorkPackage
        fields = ('id', 'project', 'work_package_name')
        import_id_fields = ('project', 'work_package_name')
        skip_unchanged = True
        report_skipped = True
        use_transactions = True

We're done. Now, go back to the admin UI and try again:

100 Work Packages have been successfully added to the database. Let's do a quick check on the Project table:

The fruit has finally arrived! Simple, right? That's the true beauty of Django in web development—very little code, yet highly efficient.

Conclusion

Django Import Export is a great library. Besides the implementation I've already introduced to you, there's much more you can explore with it. A brief overview won't disappoint you: Diango Import Export.