Deployment Instructions for `cs-webapps`

Deployment Instructions for cs-webapps
- Overview

Overview

These instructions will help you deploy your Django web application to the cs-webapps.bu.edu host to make them available on the public World-Wide-Web.

Pre-requisite Task: Setting up your Web Space

The CS Department automatically creates CS Unix accounts for all students who are registered for a CS class. The Department maintains a web server called cs-people.bu.edu.

Set up your cs-people webpage Web Space (public_html directory)

If you have done this before, you may skip this part.

In order to publish web pages on this server, you must run a short script to activate your account and set the permissions correctly. You will need to connect to the host csa2.bu.edu. Here’s how:

Open an SSH connection to the csa2.bu.edu server.

On Mac

Open the built-in Terminal app. Type the following command at the command prompt: ssh username@csa2.bu.edu

Example with Aaron’s username:

$ ssh azs@csa2.bu.edu

On Windows

You will need to install an SSH client program. I recommend that you download/install the Putty software. Open Putty. The default setting is fine for our purpose. In the prompt for Host Name (or IP address), use the hostname csa2.bu.edu and click “Open”.

Log in by typing your BU username and Kerberos password (which will not show on the screen).
After you are logged in and see the unix command prompt
(the line will end with $ or ~ or %), type the following command, and hit enter:
```
[azs@csa2 ~] $ web-ok
```

You will see an error message if it does not work, but no message if successful.

After your account is set up, go the main cs-people webpage and check that your name/account is set up. If not, ask for help!

Setting up your Django Project

The cs-webapps.bu.edu server will run your Django application, provided you follow some very specific requirements with regards to the project and file names. Failure to follow these requirements will result in your web app not being found by the Apache webserver.

Directory structure

On your local computer, you must create a directory that contains your Django project. You can name the directory anything you like, but our examples call it django, and you might as well follow that convention.

You must call your project cs412, because that is the name that the configuration file expects.

You can verify the correctness by running these commands on your local computer:
```
$ cd django
$ ls -la
total 12472
drwxr-xr-x@ 17 azs  staff      544 Nov 19 10:52 .
drwxr-xr-x@  4 azs  staff      128 Nov 19 10:52 ..
drwxr-xr-x.  7 azs  staff      224 Nov 19 09:46 cs412
-rwxr-xr-x.  1 azs  staff        0 Nov 19 11:03 db.sqlite3
drwxr-xr-x. 11 azs  staff      352 Sep 19 10:54 hw
-rwxr-xr-x.  1 azs  staff      661 Sep 12 08:34 manage.py
drwxr-xr-x.  4 azs  staff      128 Oct  1 08:56 static
```
You will have different file sizes and dates, but you should see the same basic structure. Note the cs412 directory (which contains) the project-level configuration files.

If you did not create this directory structure or did you name your project correctly, now is the time to correct it.
ALLOWED_HOSTS

Update the file cs412/settings.py to include 'cs-webapps.bu.edu' in the list of ALLOWED_HOSTS.

Static and media directories:

At the bottom of the file cs412/settings.py , use the following lines to configure the appropriate static files and media files directories. Replace username with your BU username.

STATIC_ROOT = os.path.join(BASE_DIR, 'staticfiles')
STATIC_URL = 'static/' # note: no leading slash!

STATICFILES_DIRS = [
    os.path.join(BASE_DIR, "static"),
]

MEDIA_ROOT = os.path.join(BASE_DIR, 'media/')
MEDIA_URL= "media/"  # note: no leading slash!

import socket
CS_DEPLOYMENT_HOSTNAME = 'cs-webapps.bu.edu'

if socket.gethostname() == CS_DEPLOYMENT_HOSTNAME:
    STATIC_URL = '/username/static/'
    MEDIA_URL = '/username/media/'

GitHub set up

If you do not already have an account on GitHub, go create one now. On the www.github.com website, create a private respository (repo) called cs412.

There will be a page of instructions about how to connect to this repository from your computer using command-line tools. An abbreviated version is given below:

Initialize your local Git Repository

These commands will intialize your local git repository and set some configuration parameters.

You will only need to do these steps once!

# change into Django directory
$ cd django

# create git repository, check in files
$ git init

# set git congifguration parameters
$ git config --global user.name "Your Name""
$ git config --global user.email "username@bu.edu"

Connect the remote branch (on GitHub) to your local file system

You need to connect your local git repository to the remote on GitHub.

To do so, you will need the git hook, which is a URL (ending with .git). You can obtain your git hook from the GitHub website. Log into the GitHub website, and look for a green box/dropdown labeled “Code”.

Click it to obtain the git hook URL, and then run this command:
```
# add the remote branch (from github) to the local repository
# your remote hook will be different -- get it from your github account!!
$ git remote add origin https://github.com/azs-bu/cs412-examples.git
```
When you attempt to do this, you should be prompted for a username and password. The username is your github.com username (mine is azs-bu). Check yours.
- In the password field, you must use a Personal Access Token. This is a special password that you will generate from the github website. Follow the directions on the github website (they have screenshots you can follow).
- Set the expiration to be at least 90 days so you do not need to change it for the rest of the semester.
Your should only need to do this once!
Push local changes to the remote branch

These commands will check the status of your local branch, add changes, commit them to the repository, and push them to the remote branch on Github. You will need to do these steps every time you want to deploy.
```
# check status of repository
$ git status

# add all local changes to repository 
$ git add -A

# commit changes
$ git commit -m 'input some useful comment here'

# push current repository to Github remote
$ git push origin main
```

Configuration on the CS Computing Cluster

The following steps will be required only once, to set up your webapps directory and connect your git repository.

Open a command-line prompt to csa2.bu.edu. Replace username with your BU Kerberos username in the command below:
```
$ ssh username@csa2.bu.edu
```
At the local command prompt on csa2, create a sub directory off your home directory called webapps.
```
[username@csa2 ~] $ mkdir webapps
```
Set the permissions on the webapps directory:
```
[username@csa2 ~] $ chmod 701 webapps
```
Change into the webapps directory:
```
[username@csa2 ~] $ cd webapps
```
This will cause the prompt to change (indicating the new directory):
```
[username@csa2 webapps] $
```
Create a clone of your git repository from Github to the webapps directory. To do so, you will need the git hook, which is a URL (ending with .git). You can obtain your git hook from the GitHub website. Log into the GitHub website, and look for a green box/dropdown labeled “Code”.

Click it to obtain the git hook URL.

Run the git clone command, substituting in your git hook:
```
[username@csa2 webapps] $ git clone https://github.com/username/example.git
```
You will be prompted to authenticate to GitHub (log in) using your GitHub credentials, which should be your GitHub account name and Personal Access Token (in place of password).

The git clone command will create a local copy of your GitHub repository within your CS account’s ~/webapps directory.

The webserver configuration file expects to find your directory under:
```
/home/ugrad/username/webapps/django/
```
And will not work if it cannot find your directory/files. Check the name created by the git clone command:
```
[username@csa2 webapps] $ ls -la        
drwxr-xr-x@ 17 azs  staff      544 Nov 19 10:52 .
drwxr-xr-x@  4 azs  staff      128 Nov 19 10:52 ..
drwxr-xr-x.  7 azs  staff      224 Nov 19 09:46 412-cs-webapps
```
Notice that the directory is called 412-cs-webapps (not django). This directory name corresponds to the name of your git repository; your directory might have a different name, and that’s OK! Rename the directory to django:
```
[username@csa2 webapps] $ mv 412-cs-webapps django
```
Check again:
```
[username@csa2 webapps] $ ls -la        
drwxr-xr-x@ 17 azs  staff      544 Nov 19 10:52 .
drwxr-xr-x@  4 azs  staff      128 Nov 19 10:52 ..
drwxr-xr-x.  7 azs  staff      224 Nov 19 09:46 django
```
It is crucial that you check the name of this directory and rename it to django.
Create a link to static files.

This section refers to a static directory, but you might not have created it yet in your local files (on your computer) or on the git repository; it’s part of the examples for week 3. You can skip this step and come back to it after doing the examples for week 3.

In deployment, we need to create a simbolic link to tell the webserver where to find some of the static files for the Django admin. Run these commands at the shell on csa2.bu.edu, inside your webapps directory.
```
[username@csa2 ~] $  cd webapps
[username@csa2 webapps] $ cd django
[username@csa2 django] $ cd static
[username@csa2 static] $ ln -s ../staticfiles/admin admin
```
Check that the link is correct. Run a listing:
```
[username@csa2 static] $ ls -la
```
and you should see something like this:
```
total 28
drwx-----x.  2 azs cs-faculty 4096 Nov 12 13:33 .
drwxr-xr-x. 21 azs cs-faculty 4096 Jan 26 10:13 ..
lrwxrwxrwx.  1 azs cs-faculty   20 Oct 22 14:45 admin -> ../staticfiles/admin
-rw-r--r--.  1 azs cs-faculty 3899 Nov  1 09:47 styles.css
```
Notice the pointer, linking admin to ../staticfiles/admin.

Deployment

The following steps will be required each time you want to redeploy your work to cs-webapps.bu.edu.

Verify that your Django project runs without errors on your localhost.

$ cd django
$ pipenv shell
(django) $ python manage.py runserver

The Django development server should come up with no errors:

Watching for file changes with StatReloader
Performing system checks...

System check identified no issues (0 silenced).
January 09, 2025 - 15:12:54
Django version 4.2.16, using settings 'cs412.settings'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C

If you have any errors, fix them now before continuing the deployment process.

Run the collectstatic command. This is necessary if (but only when) you add/change any of the static resources (e.g., .css, images, etc.).
```
(django) $ python manage.py collectstatic
```

Add your files and push your files to GitHub.

(django) $ git add -A
(django) $ git commit -m "Added all files for deployment."
(django) $ git push origin main

Open a command-line prompt to csa2.bu.edu. Replace username with your BU Kerberos username in the command below:
```
(django) $ ssh username@csa2.bu.edu
```
Change into the webapps/django directory:
```
[username@csa2 ~] $ cd webapps/django
```
Pull your files from the GitHub repository:
```
[username@csa2 django] $ git pull
```
To access your most recent files, the Apache webserver must be restarted. It is scheduled to restart periodically, perhaps every 5 minutes. You will need to wait several minutes to see your deployed web application, but it will be there.
If you have an error about unstaged changes

If you have an error that looks like this:
```
[username@csa2 django]$ git pull
Updating 4868edc..8efbbb3
error: Your local changes to the following files would be overwritten by merge:
        db.sqlite3
Please commit your changes or stash them before you merge.
Aborting
```
It is likely because you have made some changes that affected the Django database file (db.sqlite3). git does not want to overwrite these changes with the version from GitHub, but you probably do want to overwrite these changes. If you are not sure, talk about it with the teaching staff before you proceed.

To correct the error, you can revert to the previously committed version of a file using this command:
```
[username@csa2 django] git checkout <filename>
```
for example:
```
[username@csa2 django] git checkout db.sqlite3
```
And thereafter you can try the git pull again.
Published web application URL:

Your public web application will be visible using a URL like this:
```
https://cs-webapps.bu.edu/username/
```
(replace username with your username).

Debugging

When you see errors, it is probably due to some unresolved error that you should have corrected while testing on your localhost development server.

You can view error messages (filtered for your username/ip address) using this logview tool.

Installed Packages

The cs-webapps server has a wide range of packages installed, but not everything. If you think you need some package that is not installed, please email Aaron (azs@bu.edu) with requests.

Package              Version
-------------------- -----------
aiofiles             23.1.0
asgiref              3.7.2
beautifulsoup4       4.10.0
certifi              2023.5.7
chardet              4.0.0
charset-normalizer   2.0.10
cockpit              323.1
commonmark           0.9.1
configshell-fb       1.1.30
dasbus               1.4
dbus-python          1.2.18
decorator            4.4.2
distro               1.5.0
dj-database-url      0.5.0
Django               3.2.25
django-columns       0.1.0
django-countries     7.6.1
enrich               1.2.7
environ              1.0
et-xmlfile           1.1.0
file-magic           0.4.0
gpg                  1.15.1
gunicorn             21.2.0
idna                 2.10
iniparse             0.4
iotop                0.6
jdcal                1.4.1
kmod                 0.1
libcomps             0.1.18
localflavor          1.9
lxml                 4.6.5
mod-wsgi             4.7.1
nftables             0.1
numpy                2.0.2
olefile              0.46
openpyxl             3.0.10
packaging            20.9
pandas               2.2.3
perf                 0.1
pexpect              4.8.0
Pillow               10.0.1
pip                  21.3.1
plotly               5.24.1
psutil               5.8.0
ptyprocess           0.6.0
Pygments             2.7.4
PyGObject            3.40.1
pyinotify            0.9.6
pyparsing            2.4.7
PySocks              1.7.1
python-dateutil      2.9.0.post0
python-linux-procfs  0.7.3
pytz                 2021.1
pyudev               0.22.0
PyYAML               5.4.1
requests             2.25.1
rich                 13.1.0
rpm                  4.16.1.3
rtslib-fb            2.1.76
scipy                1.13.1
selinux              3.6
sepolicy             3.6
setools              4.4.4
setroubleshoot       3.3.32
setuptools           53.0.0
six                  1.15.0
sos                  4.8.2
soupsieve            2.4.1
sqlparse             0.4.2
subprocess-tee       0.3.5
subscription-manager 1.29.42
systemd-python       234
targetcli-fb         2.1.57
tenacity             9.0.0
typing_extensions    4.12.2
tzdata               2024.2
urllib3              1.26.5
urwid                2.1.2
whitenoise           6.1.0

Do you have suggestions for improving these instructions?

Please share them with Aaron.

Deployment Instructions for cs-webapps

Overview

Pre-requisite Task: Setting up your Web Space

Setting up your Django Project

GitHub set up

Configuration on the CS Computing Cluster

Deployment

Debugging

Installed Packages

Deployment Instructions for `cs-webapps`