when embedded with FastAPI is there a
when embedded with FastAPI is there a different port that Solara loads or it's the same port as uvicorn?
11 Replies
It's all on the same port, do you have an error in the JS console?
Yes, on Console it throws error as WebSocket Connection to wss://solara/jupyter/api/kernels failed
even without a reverse proxy?
no further details in the error?
yes, no further details, I could see the POST method in the container and when creating websocket connection is when it's failing.
_createSocket @ default.js:73
I'm using port 8080 for uvicorn, will any of jupyter kernel widgets or solara use this port internally?
It’s all on the same port, we just ride in whatever starlette is using, no different port
Hi @MaartenBreddels , just to understand why does Solara try to create a websocket connection? isn't the communication protocol HTTP/HTTPS?
This worked and the solution was firewall issue in production in which we had to allow wss connection
@MaartenBreddels thanks for helping on this issue. In production the server disconnects every now and then during execution and below is the pod logs
INFO: 10.141.12.1:45012 - "HEAD / HTTP/1.1" 200 OK
INFO: ('10.141.12.1', 45002) - "WebSocket /jupyter/api/kernels/d89f2b18-9ecb-41b8-909c-d258f2f7e56b/channels?session_id=c4304626-18b5-4778-bff7-0a41498d23a1" [accepted]
INFO: connection open
INFO: connection closed
INFO: connection closed
INFO: 10.141.4.1:49444 - "HEAD / HTTP/1.1" 200 OK
it uses a websocket connection because changes can come from the server, so it can push changes to the client
does it not reconnect?
attaching the code for your reference
@sl.component
def Page():
MyHome()
prev_text, set_prev_text = sl.use_state("") text = sl.use_reactive("") content, set_content = sl.use_state("") loading, set_loading = sl.use_state(False)
def predictions(): if text.value.strip() != "" and text.value != prev_text: set_prev_text(str(text.value)) try: set_loading(True) param = {"msg": str(text.value.strip())} res = requests.get(fhttp://0.0.0.0:8080/predict_response, params = param, verify=False) sl.Markdown(res.text) set_content(res.text) set_loading(False) except Exception as e: sl.Markdown(e)
def clear(): set_content("") text.set("") set_prev_text("")
def copy_to_clipboard(): pyperclip.copy(content) sl.Style(css)
with sl.Card(margin=30): with sl.Column(margin=30, align="start"): with sl.Row(): sl.Markdown(ner_description)
with sl.Card(margin=30): sl.InputText("Enter Input Text", value=text, continuous_update=True) with sl.Column(margin=30, align="center"): with sl.Row(justify="space-around"): sl.Button("Execute", color="green", on_click = predictions) sl.Button("Clear", color="primary", on_click = clear, on_value = prev_text)
if loading: with sl.Column(margin=30, align="center"): with sl.Row(justify="space-around"): sl.SpinnerSolara(size="200px") elif content != "": with sl.Card(margin=30): sl.Markdown("NER Model Response: ") with sl.Row(): sl.HTML(style= "margin: auto; font-size: 16px; color: black;", classes=["body"], unsafe_innerHTML=content) with sl.Card(margin=20): with sl.Column(margin=30, align="center"): with sl.Row(justify="space-around"): sl.Button("Copy To Clipboard", color="#d14d4d", on_click=copy_to_clipboard) no it takes longer and doesn't reconnect and I manually have to refresh the page. Is there a predefined timeout set on Solara UI that checks for server response? I see the server processed the input and by the time it sends back the server disconnects
MyHome()
prev_text, set_prev_text = sl.use_state("") text = sl.use_reactive("") content, set_content = sl.use_state("") loading, set_loading = sl.use_state(False)
def predictions(): if text.value.strip() != "" and text.value != prev_text: set_prev_text(str(text.value)) try: set_loading(True) param = {"msg": str(text.value.strip())} res = requests.get(fhttp://0.0.0.0:8080/predict_response, params = param, verify=False) sl.Markdown(res.text) set_content(res.text) set_loading(False) except Exception as e: sl.Markdown(e)
def clear(): set_content("") text.set("") set_prev_text("")
def copy_to_clipboard(): pyperclip.copy(content) sl.Style(css)
with sl.Card(margin=30): with sl.Column(margin=30, align="start"): with sl.Row(): sl.Markdown(ner_description)
with sl.Card(margin=30): sl.InputText("Enter Input Text", value=text, continuous_update=True) with sl.Column(margin=30, align="center"): with sl.Row(justify="space-around"): sl.Button("Execute", color="green", on_click = predictions) sl.Button("Clear", color="primary", on_click = clear, on_value = prev_text)
if loading: with sl.Column(margin=30, align="center"): with sl.Row(justify="space-around"): sl.SpinnerSolara(size="200px") elif content != "": with sl.Card(margin=30): sl.Markdown("NER Model Response: ") with sl.Row(): sl.HTML(style= "margin: auto; font-size: 16px; color: black;", classes=["body"], unsafe_innerHTML=content) with sl.Card(margin=20): with sl.Column(margin=30, align="center"): with sl.Row(justify="space-around"): sl.Button("Copy To Clipboard", color="#d14d4d", on_click=copy_to_clipboard) no it takes longer and doesn't reconnect and I manually have to refresh the page. Is there a predefined timeout set on Solara UI that checks for server response? I see the server processed the input and by the time it sends back the server disconnects
my guess this has to do with your firewall or reverse proxy. I assume you don't get this on localhost right?
and do you have load balancing?
yes, I do think the same as well, we do have load balancing, but I notice that when we pass small text as input to model it works perfect but it's only when processing large text (around 1000 words) is when it's breaking and disconnecting. The model is converted to ONNX and is placed within the container itself
you might want to enable sticky session, for a reconnect to reach the same node.
However, that is seperate from your issue. I wonder if the infrastructure puts limits on the websocket connection, like some limitation on the message size