Wordpress on Raspberry Pi - Solving the page not found issue when using permalink

This post is about how to solve the page not found problem when using the Permalinks feature of Wordpress.

My Environment

pi@raspberrypi:/ $ lsb_release -a
No LSB modules are available.
Distributor ID: Raspbian
Description:    Raspbian GNU/Linux 9.9 (stretch)
Release:        9.9
Codename:       stretch
pi@raspberrypi:/ $