Creating a TimescaleDB Hypertable
Hi y'all, I'm creating a prisma migration for an existing table on Neon
the prisma schema is
I'm getting a
column ""createdAt"" does not exist
What am I missing??
17 Replies
fascinating-indigo•2y ago
Can you dump the schema of your database and see if your table even exists with the columns you expect
deep-jadeOP•2y ago
@Tristan Partin I checked, it does and the case is consistant with schema definition
@Tristan Partin If you create a table following the schema mentioned above - are your able to make it a hypertable?
fascinating-indigo•2y ago
@zatmonkey looking at it again, what happens if you remove the inner double quotes?
I have never used timescale
deep-jadeOP•2y ago
if i remove them on FundHistory, then it does not find the table, on created at it does not make a difference @Tristan Partin
fascinating-indigo•2y ago
If you run this migration on a local postgres instance, does it work?
deep-jadeOP•2y ago
haven't tried this i'd need to spin one up
i mean i was looking for answers, not step to investigate
harsh-harlequin•2y ago
The right syntax is :
"select create_hypertable('"FundHistory"','createdAt');"
I took ownership on the case you raised this morning and was replying to you via email 🙂
Simply put, the create_hypertable function calls a c function which handle the table name and the column name differently
I assume that you saw this article: https://www.timescale.com/forum/t/pass-case-sensitive-view-name-to-timescaledb-functions/1021
This information is however incorrect.
If you look at the source code of the create_hypertable function (https://github.com/timescale/timescaledb/blob/main/sql/ddl_api.sql#L23-L38), you will notice that it pass the parameters to a c function named
ts_hypertable_create (https://github.com/timescale/timescaledb/blob/main/src/hypertable.c#L1647), itself calling ts_hypertable_create_time_prev (https://github.com/timescale/timescaledb/blob/main/src/hypertable.c#L1587 ).
At the contrary of what the timescale community post says, only the relation name is casted as REGCLASS.
The column name is instead cast as ::NAME, which eventually translate into:
Oid table_relid = PG_ARGISNULL(0) ? InvalidOid : PG_GETARG_OID(0);
and
Name open_dim_name = PG_ARGISNULL(1) ? NULL : PG_GETARG_NAME(1);`
So, using '" "' (single quote - double quote) does work for relation name, but doesn't for column names.
For columns names, it appears that a simple single quote is sufficient, even if the column name is case sensitive
Here is a quick reproduction from one of my test projects to demonstrate the above:
select create_hypertable("FundHistory",'createdAt'); -- ERROR: column "FundHistory" does not exist (SQLSTATE 42703) select create_hypertable("FundHistory","createdAt"); -- ERROR: column "FundHistory" does not exist (SQLSTATE 42703) select create_hypertable('FundHistory',"createdAt"); -- ERROR: column "createdAt" does not exist (SQLSTATE 42703) select create_hypertable('FundHistory','createdAt'); -- ERROR: relation "fundhistory" does not exist (SQLSTATE 42P01) select create_hypertable('"FundHistory"',"createdAt"); -- ERROR: column "createdAt" does not exist (SQLSTATE 42703) select create_hypertable('"FundHistory"','"createdAt"'); -- ERROR: column ""createdAt"" does not exist (SQLSTATE 42703) select create_hypertable('"FundHistory"','"createdAt"'::NAME); -- ERROR: column ""createdAt"" does not exist (SQLSTATE 42703) select create_hypertable('"FundHistory"','createdAt'); -- (3,public,FundHistory,t)Additionally, please note that due to licensing restrictions, it's not possible to use compression with the TimescaleDB extension in Neon. Timescale provides this extension with two different licences: The Apache 2 Edition and the Community Edition. The first one is allowed for commercial uses, while the second is not. Unfortunately, the Apache 2 Edition doesn't include features provided by the Community Edition. See: https://docs.timescale.com/about/latest/timescaledb-editions/
xenophobic-harlequin•2y ago
Not everyone in here is an expert on every technology. Please keep it respectful.
Thanks Yanic and Tristan!
fascinating-indigo•2y ago
@Yanic great investigation!
Thanks for confirming my suspicion
deep-jadeOP•2y ago
sorry @Tristan Partin i wasn't trying to be disrespectful - somtimes chats come out dry
@Yanic thanks for the email - super helpful
harsh-harlequin•2y ago
With a real pleasure @zatmonkey !
deep-jadeOP•2y ago
(and impressed by the deep dig :D)
harsh-harlequin•2y ago
Honestly, this seemed as weird to me that it did for you!
I was really curious to get to the bottom of it, because it doesn't make much sense when you first hit the issue :p
deep-jadeOP•2y ago
alright, there were some additional secondary issues but I finally got it working thanks @Yanic
I'll assume compression is not critical for now. I'll think to add an aggregation table at some point I guess Hopefully you'll offer compression as an option in the future - looking more into it that's were the biggest value lies in this extension
I'll assume compression is not critical for now. I'll think to add an aggregation table at some point I guess Hopefully you'll offer compression as an option in the future - looking more into it that's were the biggest value lies in this extension
fascinating-indigo•2y ago
The license of the timescale extension prohibits that. You might be able to bring your own extension though. You would have to inquire about that
The other option is using neon and timescale together
deep-jadeOP•2y ago
or y'all can offer it as an add on and rev-share with timescale
fascinating-indigo•2y ago
right, I'll provide your feedback internally