Python project structure

By the end of this blog post, you will be able to:- Create a well-structured Python project
- Use relative and absolute imports in a Python project
- Invoke a specific module from the command-line (terminal) Link to GitHub repository - [unfor19/python-project](https://github.com/unfor19/python-project)

python-project

Python project structure, relative imports, absolute imports, packages, and modules. Let's make it simpler.

Getting Started

Executing modules from the project's root directory (top-level package)

# main.py

‍

# app.py

‍

message.py

Questions and Answers (Q&A)

Project, Packages and Modules, what are they?

• Project - a directory, also known as the top-level package, which contains packages and modules
• Package (in a project) - a directory which contains modules and/or packages (sub-directories)
• Module - a Python script (`.py`) which can be expected from the terminal, or imported with `import` and `from`

What about Packages which are not part of a project?

• Package (built-in) - a package which is shipped with Python and can be imported with `import` and `from
• Package (pip) - a package which is installed with pip and can be imported with `import` and `from`. Think about it, pip stands for Package Installer for Python

How do I import Packages and Modules that I've created?

• Python project's packages and modules can be imported with **relative paths** from any module which is **part of the same project**. An example is available in src/app/app.py
• If you intend to import a package or a module which is **not part of the same project**, you'll have to use **absolute paths**. This can be done with importlib see this StackOverflow answer

‍

‍Do I need the __init__.py file?

• Short answer - no
• In previous versions of Python, you had to create the `__init__.py` file in each directory that you want to import as a package, they were called _regular packages_. From version 3.3+ it is not required anymore - Implicit Namespace Packages packages without an `__init__.py` file are called _namespace packages

Why do relative imports raise a problem in pylint?

The error - Attempted relative import beyond top-level packagepylint(relative-beyond-top-level)

• Short answer - I don't know‍
• All I can say is that it doesn't happen with flake8

Is it possible to invoke a function from the terminal?

• Short answer - it depends
• Trying to invoke a function from the terminal, such as `src.app.app.main()`, will raise the ModuleNotFound exception. A package must be imported before invoking one of its functions.  

• Since you can't invoke `main()` directly from the terminal, calling it from the `if __main__` block enables executing it from the terminal. It's possible to pass arguments, but it's a bit ugly, read the docs to learn how. The following example attempts to execute the module `src.app.app`, which in turn call its `if __main__` block

• If the PWD is a subdirectory of the project, such as python-project/appy, an attempt to execute a module which contains relative imports, will raise the exception below. Remember, your PWD should always be the project's root directory, in this case it's python-project

• It doesn't happen when invoking message, since message doesn't use relative imports

• Invoking a function from the terminal is also possible by using the -c flag. Surprise, it's possible to pass arguments in a more intuitive way, for example app.main(my_arg1, my_arg2)

‍

What are the available command-line flags in Python?

• Read the docs - using cmdline
• In this tutorial, we used both -c and -m flags

Why is it possible to execute python -m appy?

The appy/__main__.py file acts like the if __main__ code snippet, but on packages. This enables the appy package to be executed with python -m or with runpy

‍

What's runpy and why do you use it in main.py?

The runpy package provides the ability to run modules from a module (Python script).

main.py

Why do you have a weird path with pyenv when you run Python?

In some of the examples you might have seen that my Python binary is located in

This is because I'm using pyenv, the official definition from the docs

pyenv lets you easily switch between multiple versions of Python. It's simple, unobtrusive, and follows the UNIX tradition of single-purpose tools that do one thing well.

• pyenv great for checking backwards compatibility
• Switching to a different version

1. Install relevant version - pyenv install 3.7.7
2. Run export PYENV_VERSION=3.7.7

• For a day to day use

1. Install relevant version - pyenv install 3.8.2
2. Add export PYENV_VERSION=3.8.2 to your terminal's rc or _profile ($HOME/.bashrc, $HOME/.bash_profile, $HOME/.zshrc)

Examples - Expand/Collapse

Is there a good framework for creating a Python CLI?

• Short answer - yes! more than a few!
• The Click Framework is amazing! I used it in githubsecrets and frigga NOTE: you'll discover that I have __init__.py files in my projects, that's a nasty old habit, I promise I will drop them, I promise
• Read more about the available Python CLI frameworks in this great blog post - Building Beautiful Command Line Interfaces with Python

Where can I find a well-structured Python project?

• sampleproject - the official Python sample project
• Django - a complex Python project
• requests - a non-complex Python project

Final words

I hope that this blog post helped you in understanding how to create a well-structured Python project, and if it did, then Heart, Clap, Star and share!