Self hosted Novu with Firebase provider error

Soumith · 2024-09-07T15:03:21.564Z

Hi Team, We are using Firebase provider for push notifications on self hosted novu, and in our production environment we see this error when we trigger notifications on novu activity: "Sending message failed due to \"Operation is not implemented, or supported, or enabled. Raw server response: \"{\"error\":{\"code\":501,\"message\":\"Operation is not implemented, or supported, or enabled.\",\"status\":\"UNIMPLEMENTED\"}}\"\"" Could you guys please shed some light on this error? what could be causing this?

S

SoumithOP•9/7/24, 5:26 PM

https://github.com/firebase/firebase-admin-node/issues/2687

https://groups.google.com/g/firebase-talk/c/1OHTEwSzZYU

https://stackoverflow.com/questions/78894668/phone-registration-error-unimplemented-gcm-error-on-a-device-using-javascript

I found similar issues for older Firebase SDK versions

GitHub

messaging/unknown-error when using sendEachForMulticast · Issue #26...

Operating System version: AWS Lambda Firebase SDK version: firebase-admin 12.3.0 (but also previous versions) Firebase Product: message notification Node.js version: Node.JS 18 NPM version: 9 I hav...

Stack Overflow

PHONE_REGISTRATION_ERROR / UNIMPLEMENTED GCM error on a device usin...

We are using devices which has Linux OS, javascript enabled (using node.js). We are using google FCM to receive notifications. We are using a javascript library (added register() here) to get the fcm

N

Novu_Bot•9/7/24, 5:26 PM

@Soumith, you just advanced to level 2!

P

Pawan Jain•9/9/24, 7:45 AM

Hey @Soumith

This issue is happening due to deprecate

sendMultiCast

sendMultiCast

sendMultiCast

sendMultiCast method. it should be replaced with

sendEachForMulticast

sendEachForMulticast

sendEachForMulticast

sendEachForMulticast
We fixed this issue in cloud but have not released new docker images that will include this fix

S

SoumithOP•9/9/24, 1:20 PM

Hi @Pawan Jain,
Do you know when this fix will be released?

P

Pawan Jain•9/10/24, 10:14 AM

@Soumith

We are planning to release new docker images early October

F

frnandu•9/10/24, 12:22 PM

shit, this is fucked up for self-deployed instances using push notifications in production!

F

frnandu•9/10/24, 12:27 PM

I would treat this as critical thing to fix as soon as possible, since it means that push don't work anymore to fcm. And I understand that you concentrate on cloud version and self-hosted are more or less on their own, but missing a build/image for such critical fix really makes our company seriously reconsider using novu, even if self-hosted.

Ffrnandu I would treat this as critical thing to fix as soon as possible, since it means ...

S

SoumithOP•9/10/24, 1:36 PM

Agree!!!

PPawan Jain @Soumith We are planning to release new docker images early October

S

SoumithOP•9/10/24, 1:37 PM

@Pawan Jain This is breaking in production and we cannot afford customers losing notifications for a month, as this is a hotfix, can you guys please provide interim solution?

P

Pawan Jain•9/10/24, 1:58 PM

@Soumith @frnandu

We are working on a hotfix for this issue.

I will keep you updated in this post

F

frnandu•9/10/24, 2:19 PM

Thank you!

S

SoumithOP•9/10/24, 2:43 PM

Thank you @Pawan Jain!

T

Tomer Barnea•9/10/24, 9:34 PM

Hey all

the team is working hard to make sure we can release a hotfix soon. @Pawan Jain will follow up ASAP with more updates as the team roles them out.
Thanks for understanding and being an amazing community!

S

SoumithOP•9/11/24, 4:23 AM

Thanks for the update @Tomer Barnea!

E

Eduardo Santos•9/11/24, 11:54 AM

I'm having the same issue. Hope to get this hotfix soon because it's blocking my release to production.

C

Chandan kumar•9/11/24, 1:30 PM

We are looking for immediate fix...currently all push notifications stopped.

T

Tomer Barnea•9/11/24, 4:57 PM

@Pawan Jain can you please give us an update on this topic.

P

Pawan Jain•9/11/24, 5:12 PM

Small Update on this:-

We have published docker images with tag

0.24.3

0.24.3

0.24.3

0.24.3
However, there is one issue with api docker image due to which this tagged version can not be used.

P

Pawan Jain•9/11/24, 5:12 PM

Sok from our team is working on resolving that issue

P

Pawan Jain•9/11/24, 5:50 PM

@Soumith @Eduardo Santos @frnandu @Chandan kumar @Fakhar @Adil Javed

Thanks for being patient with us.

update your
```
worker
```
worker
```
worker
```
worker
and other services docker image tag to
```
 0.24.3
```
0.24.3
```
 0.24.3
```
0.24.3
except api service

if you see below error with

api

api

api

api tag

0.24.1

0.24.1

0.24.1

0.24.1 and

0.24.2

0.24.2

0.24.2

0.24.2. use API docker image tag

0.24.0

0.24.0

0.24.0

0.24.0

✘ api Error manifest unknown                                                                    3.2s 
Error response from daemon: manifest unknown

✘ api Error manifest unknown                                                                    3.2s 
Error response from daemon: manifest unknown

✘ api Error manifest unknown                                                                    3.2s 
Error response from daemon: manifest unknown

✘ api Error manifest unknown                                                                    3.2s 
Error response from daemon: manifest unknown

P

Pawan Jain•9/11/24, 5:51 PM

checkout this file changes
https://github.com/novuhq/novu/pull/6488/files#diff-3553fc766756fb977edd9560e25cd13d2f8e0eb268a1f91760ad298482766b75

S

SoumithOP•9/11/24, 8:40 PM

Thank you so much @Pawan Jain! We'll test and let you know

PPawan Jain checkout this file changes https://github.com/novuhq/novu/pull/6488/files#diff-3...

S

SoumithOP•9/11/24, 8:47 PM

Please let us know once this is merged.

G

GetWrong•9/12/24, 5:48 AM

hope this new docker image release soon

SSoumith Please let us know once this is merged.

P

Pawan Jain•9/12/24, 6:22 AM

You can go ahead manually change the version in your docker-compose.yml file

GGetWrong hope this new docker image release soon

P

Pawan Jain•9/12/24, 6:22 AM

@GetWrong

We released 0.24.3 version.

E

Eduardo Santos•9/12/24, 7:10 AM

Thanks @Pawan Jain , I'll talk with my team here.

PPawan Jain @GetWrong We released 0.24.3 version.

K

Katya Kuzmenchuk•9/12/24, 8:16 AM

Hi @Pawan Jain I've encountered an issue while running novu-api 0.24.3. Please find stack trace below

2024-09-12 10:04:24 Error: Cannot find module '@novu/shared/utils'
2024-09-12 10:04:24 Require stack:
2024-09-12 10:04:24 - /usr/src/app/node_modules/.pnpm/@novu+node@0.24.2/node_modules/@novu/node/build/main/lib/novu.js
2024-09-12 10:04:24 - /usr/src/app/node_modules/.pnpm/@novu+node@0.24.2/node_modules/@novu/node/build/main/index.js
2024-09-12 10:04:24 - /usr/src/app/apps/api/dist/main.js
2024-09-12 10:04:24     at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1225:15)
2024-09-12 10:04:24     at Module.Hook._require.Module.require (/usr/local/lib/node_modules/pm2/node_modules/require-in-the-middle/index.js:81:25)
2024-09-12 10:04:24     at require (node:internal/modules/helpers:179:18)
2024-09-12 10:04:24     at Object.<anonymous> (/usr/src/app/node_modules/.pnpm/@novu+node@0.24.2/node_modules/@novu/node/build/main/lib/novu.js:34:15)
2024-09-12 10:04:24     at Module._compile (node:internal/modules/cjs/loader:1469:14)
2024-09-12 10:04:24     at Object.Module._extensions..js (node:internal/modules/cjs/loader:1548:10)
2024-09-12 10:04:24     at Module.load (node:internal/modules/cjs/loader:1288:32)
2024-09-12 10:04:24     at Function.Module._load (node:internal/modules/cjs/loader:1104:12)
2024-09-12 10:04:24     at Module.require (node:internal/modules/cjs/loader:1311:19)
2024-09-12 10:04:24     at Module.Hook._require.Module.require (/usr/local/lib/node_modules/pm2/node_modules/require-in-the-middle/index.js:101:39)
2024-09-12 10:04:24     at require (node:internal/modules/helpers:179:18)
2024-09-12 10:04:24     at Object.<anonymous>

2024-09-12 10:04:24 Error: Cannot find module '@novu/shared/utils'
2024-09-12 10:04:24 Require stack:
2024-09-12 10:04:24 - /usr/src/app/node_modules/.pnpm/@novu+node@0.24.2/node_modules/@novu/node/build/main/lib/novu.js
2024-09-12 10:04:24 - /usr/src/app/node_modules/.pnpm/@novu+node@0.24.2/node_modules/@novu/node/build/main/index.js
2024-09-12 10:04:24 - /usr/src/app/apps/api/dist/main.js
2024-09-12 10:04:24     at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1225:15)
2024-09-12 10:04:24     at Module.Hook._require.Module.require (/usr/local/lib/node_modules/pm2/node_modules/require-in-the-middle/index.js:81:25)
2024-09-12 10:04:24     at require (node:internal/modules/helpers:179:18)
2024-09-12 10:04:24     at Object.<anonymous> (/usr/src/app/node_modules/.pnpm/@novu+node@0.24.2/node_modules/@novu/node/build/main/lib/novu.js:34:15)
2024-09-12 10:04:24     at Module._compile (node:internal/modules/cjs/loader:1469:14)
2024-09-12 10:04:24     at Object.Module._extensions..js (node:internal/modules/cjs/loader:1548:10)
2024-09-12 10:04:24     at Module.load (node:internal/modules/cjs/loader:1288:32)
2024-09-12 10:04:24     at Function.Module._load (node:internal/modules/cjs/loader:1104:12)
2024-09-12 10:04:24     at Module.require (node:internal/modules/cjs/loader:1311:19)
2024-09-12 10:04:24     at Module.Hook._require.Module.require (/usr/local/lib/node_modules/pm2/node_modules/require-in-the-middle/index.js:101:39)
2024-09-12 10:04:24     at require (node:internal/modules/helpers:179:18)
2024-09-12 10:04:24     at Object.<anonymous>

2024-09-12 10:04:24 Error: Cannot find module '@novu/shared/utils'
2024-09-12 10:04:24 Require stack:
2024-09-12 10:04:24 - /usr/src/app/node_modules/.pnpm/@novu+node@0.24.2/node_modules/@novu/node/build/main/lib/novu.js
2024-09-12 10:04:24 - /usr/src/app/node_modules/.pnpm/@novu+node@0.24.2/node_modules/@novu/node/build/main/index.js
2024-09-12 10:04:24 - /usr/src/app/apps/api/dist/main.js
2024-09-12 10:04:24     at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1225:15)
2024-09-12 10:04:24     at Module.Hook._require.Module.require (/usr/local/lib/node_modules/pm2/node_modules/require-in-the-middle/index.js:81:25)
2024-09-12 10:04:24     at require (node:internal/modules/helpers:179:18)
2024-09-12 10:04:24     at Object.<anonymous> (/usr/src/app/node_modules/.pnpm/@novu+node@0.24.2/node_modules/@novu/node/build/main/lib/novu.js:34:15)
2024-09-12 10:04:24     at Module._compile (node:internal/modules/cjs/loader:1469:14)
2024-09-12 10:04:24     at Object.Module._extensions..js (node:internal/modules/cjs/loader:1548:10)
2024-09-12 10:04:24     at Module.load (node:internal/modules/cjs/loader:1288:32)
2024-09-12 10:04:24     at Function.Module._load (node:internal/modules/cjs/loader:1104:12)
2024-09-12 10:04:24     at Module.require (node:internal/modules/cjs/loader:1311:19)
2024-09-12 10:04:24     at Module.Hook._require.Module.require (/usr/local/lib/node_modules/pm2/node_modules/require-in-the-middle/index.js:101:39)
2024-09-12 10:04:24     at require (node:internal/modules/helpers:179:18)
2024-09-12 10:04:24     at Object.<anonymous>

2024-09-12 10:04:24 Error: Cannot find module '@novu/shared/utils'
2024-09-12 10:04:24 Require stack:
2024-09-12 10:04:24 - /usr/src/app/node_modules/.pnpm/@novu+node@0.24.2/node_modules/@novu/node/build/main/lib/novu.js
2024-09-12 10:04:24 - /usr/src/app/node_modules/.pnpm/@novu+node@0.24.2/node_modules/@novu/node/build/main/index.js
2024-09-12 10:04:24 - /usr/src/app/apps/api/dist/main.js
2024-09-12 10:04:24     at Function.Module._resolveFilename (node:internal/modules/cjs/loader:1225:15)
2024-09-12 10:04:24     at Module.Hook._require.Module.require (/usr/local/lib/node_modules/pm2/node_modules/require-in-the-middle/index.js:81:25)
2024-09-12 10:04:24     at require (node:internal/modules/helpers:179:18)
2024-09-12 10:04:24     at Object.<anonymous> (/usr/src/app/node_modules/.pnpm/@novu+node@0.24.2/node_modules/@novu/node/build/main/lib/novu.js:34:15)
2024-09-12 10:04:24     at Module._compile (node:internal/modules/cjs/loader:1469:14)
2024-09-12 10:04:24     at Object.Module._extensions..js (node:internal/modules/cjs/loader:1548:10)
2024-09-12 10:04:24     at Module.load (node:internal/modules/cjs/loader:1288:32)
2024-09-12 10:04:24     at Function.Module._load (node:internal/modules/cjs/loader:1104:12)
2024-09-12 10:04:24     at Module.require (node:internal/modules/cjs/loader:1311:19)
2024-09-12 10:04:24     at Module.Hook._require.Module.require (/usr/local/lib/node_modules/pm2/node_modules/require-in-the-middle/index.js:101:39)
2024-09-12 10:04:24     at require (node:internal/modules/helpers:179:18)
2024-09-12 10:04:24     at Object.<anonymous>

P

Pawan Jain•9/12/24, 8:18 AM

@Katya Kuzmenchuk , use

0.24.0

0.24.0

0.24.0

0.24.0 tag for

api

api

api

api service, for all other services you can use

0.24.3

0.24.3

0.24.3

0.24.3 tag

E

Eduardo Santos•9/12/24, 8:51 AM

@Pawan Jain we're now having 2 errors for the same push step:

P

Pawan Jain•9/12/24, 8:52 AM

@Eduardo Santos

These errors are different
https://stackoverflow.com/a/56218146

Stack Overflow

"Requested entity was not found" error when trying to send a push n...

I'm trying to send a multicast notification via FCM from a Firebase Cloud function with the following code:

const message = {
tokens: recipients,
notification: {
title: title,
...

P

Pawan Jain•9/12/24, 8:53 AM

https://stackoverflow.com/a/42290231

Stack Overflow

FCM getting MismatchSenderId

I have an application that uses Google FCM for sending push notifications.

When i send a push notification to a group of users, i get a response of MismatchSenderId for some of them. Even though, ...

E

Eduardo Santos•9/12/24, 8:54 AM

But I'm able to send the push to the same device tokens directly using FCM Api

P

Pawan Jain•9/12/24, 9:06 AM

So, there could be the case you have different service.json of different project in Novu and token was generated using other fcm project

E

Eduardo Santos•9/12/24, 9:13 AM

It's working now, it might be some delay or caching.

R

rolkool•9/30/24, 9:54 AM

Hello! We just did the proposed updated above on self-hosted instances.
We get the "Requested entity was not found error" mentioned above.
@Eduardo Santos was it finally a cache issue?
It is already 30 minutes after the update and push notifications still have this error

N

Novu_Bot•9/30/24, 9:54 AM

@rolkool, you just advanced to level 1!

P

Pawan Jain•10/1/24, 10:39 PM

@rolkool

Requested entity was not found error

Requested entity was not found error

Requested entity was not found error

Requested entity was not found error error occurs if deviceToken is not generated using same credentials as same as service.json credentials

PPawan Jain @Katya Kuzmenchuk , use `0.24.0` tag for `api` service, for all other services y...

F

Fakhar•3/4/25, 9:25 AM

@Pawan Jain has team fixed this issue in api 0.24.3, Actually I'm using Novu version 0.24.3 for all services but my api is still on 0.24.0

FFakhar @Pawan Jain has team fixed this issue in api 0.24.3, Actually I'm using Novu ve...

F

Fakhar•3/5/25, 6:22 AM

@Support @Community Support

T

Tomer Barnea•3/5/25, 6:24 AM

Hi @Fakhar I was reading the thread, what exactly are looking to understand? Which issue are you referring to?

KKatya Kuzmenchuk Hi @Pawan Jain I've encountered an issue while running novu-api 0.24.3. Please f...

F

Fakhar•3/5/25, 6:28 AM

@Tomer Barnea that was the issue in novu-api 0.24.3 when 0.24.3 were released so I just want to confirm either it has been resolved or not because I am still using this version but my api is on 0.24.0 which is api stable version

P

Pawan Jain•3/5/25, 6:35 AM

@Fakhar

You can use 0.24.0 version for api service as there are not a significant changes in api service. I will try to reproduce the issue with 0.24.4 tag

PPawan Jain @Fakhar You can use 0.24.0 version for api service as there are not a signific...

F

Fakhar•3/5/25, 9:29 AM

okay thanks @Pawan Jain

Self hosted Novu with Firebase provider error

Similar Threads

Similar Threads

Similar Threads