S
Solara7mo ago
Manoj

when embedded with FastAPI is there a

when embedded with FastAPI is there a different port that Solara loads or it's the same port as uvicorn?
11 Replies
MaartenBreddels
MaartenBreddels7mo ago
It's all on the same port, do you have an error in the JS console?
Manoj
Manoj7mo ago
Yes, on Console it throws error as WebSocket Connection to wss://solara/jupyter/api/kernels failed
MaartenBreddels
MaartenBreddels7mo ago
even without a reverse proxy? no further details in the error?
Manoj
Manoj7mo ago
yes, no further details, I could see the POST method in the container and when creating websocket connection is when it's failing. _createSocket @ default.js:73 I'm using port 8080 for uvicorn, will any of jupyter kernel widgets or solara use this port internally?
MaartenBreddels
MaartenBreddels7mo ago
It’s all on the same port, we just ride in whatever starlette is using, no different port
Manoj
Manoj7mo ago
Hi @MaartenBreddels , just to understand why does Solara try to create a websocket connection? isn't the communication protocol HTTP/HTTPS? This worked and the solution was firewall issue in production in which we had to allow wss connection @MaartenBreddels thanks for helping on this issue. In production the server disconnects every now and then during execution and below is the pod logs INFO: 10.141.12.1:45012 - "HEAD / HTTP/1.1" 200 OK INFO: ('10.141.12.1', 45002) - "WebSocket /jupyter/api/kernels/d89f2b18-9ecb-41b8-909c-d258f2f7e56b/channels?session_id=c4304626-18b5-4778-bff7-0a41498d23a1" [accepted] INFO: connection open INFO: connection closed INFO: connection closed INFO: 10.141.4.1:49444 - "HEAD / HTTP/1.1" 200 OK
MaartenBreddels
MaartenBreddels7mo ago
it uses a websocket connection because changes can come from the server, so it can push changes to the client does it not reconnect?
Manoj
Manoj7mo ago
attaching the code for your reference @sl.component def Page():
MyHome()
prev_text, set_prev_text = sl.use_state("") text = sl.use_reactive("") content, set_content = sl.use_state("") loading, set_loading = sl.use_state(False)
def predictions(): if text.value.strip() != "" and text.value != prev_text: set_prev_text(str(text.value)) try: set_loading(True) param = {"msg": str(text.value.strip())} res = requests.get(fhttp://0.0.0.0:8080/predict_response, params = param, verify=False) sl.Markdown(res.text) set_content(res.text) set_loading(False) except Exception as e: sl.Markdown(e)
def clear(): set_content("") text.set("") set_prev_text("")
def copy_to_clipboard(): pyperclip.copy(content) sl.Style(css)
with sl.Card(margin=30): with sl.Column(margin=30, align="start"): with sl.Row(): sl.Markdown(ner_description)
with sl.Card(margin=30): sl.InputText("Enter Input Text", value=text, continuous_update=True) with sl.Column(margin=30, align="center"): with sl.Row(justify="space-around"): sl.Button("Execute", color="green", on_click = predictions) sl.Button("Clear", color="primary", on_click = clear, on_value = prev_text)
if loading: with sl.Column(margin=30, align="center"): with sl.Row(justify="space-around"): sl.SpinnerSolara(size="200px") elif content != "": with sl.Card(margin=30): sl.Markdown("NER Model Response: ") with sl.Row(): sl.HTML(style= "margin: auto; font-size: 16px; color: black;", classes=["body"], unsafe_innerHTML=content) with sl.Card(margin=20): with sl.Column(margin=30, align="center"): with sl.Row(justify="space-around"): sl.Button("Copy To Clipboard", color="#d14d4d", on_click=copy_to_clipboard) no it takes longer and doesn't reconnect and I manually have to refresh the page. Is there a predefined timeout set on Solara UI that checks for server response? I see the server processed the input and by the time it sends back the server disconnects
MaartenBreddels
MaartenBreddels7mo ago
my guess this has to do with your firewall or reverse proxy. I assume you don't get this on localhost right? and do you have load balancing?
Manoj
Manoj7mo ago
yes, I do think the same as well, we do have load balancing, but I notice that when we pass small text as input to model it works perfect but it's only when processing large text (around 1000 words) is when it's breaking and disconnecting. The model is converted to ONNX and is placed within the container itself
MaartenBreddels
MaartenBreddels7mo ago
you might want to enable sticky session, for a reconnect to reach the same node. However, that is seperate from your issue. I wonder if the infrastructure puts limits on the websocket connection, like some limitation on the message size