Installing SSL fastapi as a reverse proxy on Litespeed Server

Required installations

For this, of course, we will first install a fastapi service. Linux-based Ubuntu would be a good choice for this. First, we need to do the basic installations on the terminal screen of the operating system;

pip install fastapi streamlit ctranslate2 sentencepiece transformers uvicorn

On the other hand, we need to choose a domain that will provide us with an SSL connection. After completing the website setup procedures, which is another topic, for example, if a translation service is to be established, let’s create a “translator_app” folder inside the “public_html” folder within the website and create the “main.py” python file that will execute the API functions;

Fastapi python service installation

from fastapi import FastAPI, Request
from pydantic import BaseModel
import ctranslate2
from transformers import MarianTokenizer

# Load tokenizer and model
tokenizer = MarianTokenizer.from_pretrained("Helsinki-NLP/opus-mt-tr-ar")
translator = ctranslate2.Translator("ctranslate", device="cpu")

app = FastAPI()

class TranslateRequest(BaseModel):
    text: str

@app.post("/translate")
def translate(req: TranslateRequest):
    # Encode input and convert to tokens
    encoded = tokenizer.prepare_seq2seq_batch([req.text], return_tensors="pt")
    input_tokens = tokenizer.convert_ids_to_tokens(encoded['input_ids'][0])
    
    # Run translation
    result = translator.translate_batch([input_tokens], beam_size=5)
    output_tokens = result[0].hypotheses[0]
    
    # Decode output
    output_ids = tokenizer.convert_tokens_to_ids(output_tokens)
    translated = tokenizer.decode(output_ids, skip_special_tokens=True)
    return {"translation": translated}

Of course, the “ctranslate” folder should be created in this folder and the model “bin” file, tokenizers and other necessary dictionary files should be added to it.

Operating system settings

In addition, in order for this to work as a service, the service is created with the following command and the following settings are entered, then the necessary commands are entered to make it a permanent service and start over with every restart.

 sudo nano /etc/systemd/system/translator-api.service

[Unit]
Description=Translation API Service
After=network.target

[Service]
User=apias3927
WorkingDirectory=/home/api.domain.com/public_html/translator_app
ExecStart=/usr/local/bin/uvicorn main:app --host 0.0.0.0 --port 8000
Restart=always

[Install]
WantedBy=multi-user.target

sudo systemctl start translator-api   
sudo systemctl status translator-api   
sudo systemctl daemon-reload

sudo systemctl daemon-reexec

For testing, port 8000 is checked with the curl function.

curl -X POST http://localhost:8000/translate -H "Content-Type: application/json" -d '{"text":"Hello world!"}'

sudo lsof -i :8000

COMMAND   PID      USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
uvicorn 43013 username    7u  IPv4 765098      0t0  TCP *:8000 (LISTEN)

Up to this point everything is fine and good, but there is a problem. The “http://localhost:8000/translate” call is not an SSL protocol, that is, the “https://api.domain.com/translate” that we will use to reach from outside is not ready yet. A reverse proxy must be set up for this, but the manuals are generally explained via nginx. So how will this be done via litespeed server or openlitespeed?

Reverse proxy on Openlitespeed

http://ip:7080

Server Configuration > External App > Add
Web Server
isim fastapi_backend
adres http://127.0.0.1:8000
Max Connections * 100
Initial Request Timeout (secs) * 30
Retry Timeout (secs) * 0
Response Buffering * No

vHost Config

And Finally vHost Config. The most critical part which all the traffic relay on SSL

docRoot                   /home/api.domain.com/public_html/translator_app
vhDomain                  $VH_NAME
vhAliases                 www.$VH_NAME
adminEmails               email@gmail.com
enableGzip                1
enableIpGeo               0

errorlog logs/$VH_NAME.error_log {
  logLevel WARN
  rollingSize 10M
}

accesslog logs/$VH_NAME.access_log {
  rollingSize 10M
  keepDays 30
}

rewrite  {
 enable                  1
  autoLoadHtaccess        1
rules        <<<END_rules
 REWRITERULE ^(.*)$ http://fastapi_backend/$1 [P]
END_rules
}


vhssl  {
  keyFile                 /etc/letsencrypt/live/api.domain.com/privkey.pem
  certFile                /etc/letsencrypt/live/api.domain.com/fullchain.pem
  certChain               1
  sslProtocol             24
  enableECDHE             1
  renegProtection         1
  sslSessionCache         1
  enableSpdy              15
  enableStapling           1
  ocspRespMaxAge           86400
}