N
Neon•2y ago
deep-jade

Creating a TimescaleDB Hypertable

Hi y'all, I'm creating a prisma migration for an existing table on Neon
-- This is an empty migration.

-- Enable TimescaleDB
CREATE EXTENSION IF NOT EXISTS timescaledb;


-- Add TimescaleDB-specific commands below
SELECT create_hypertable('"FundHistory"', '"createdAt"', migrate_data => true);

-- Enable compression
ALTER TABLE '"FundHistory"' SET (timescaledb.compress, timescaledb.compress_segmentby = 'fundId');

-- Add a compression policy
SELECT add_compression_policy('"FundHistory"', INTERVAL '7 days');
-- This is an empty migration.

-- Enable TimescaleDB
CREATE EXTENSION IF NOT EXISTS timescaledb;


-- Add TimescaleDB-specific commands below
SELECT create_hypertable('"FundHistory"', '"createdAt"', migrate_data => true);

-- Enable compression
ALTER TABLE '"FundHistory"' SET (timescaledb.compress, timescaledb.compress_segmentby = 'fundId');

-- Add a compression policy
SELECT add_compression_policy('"FundHistory"', INTERVAL '7 days');
the prisma schema is
// Tracks the history for a Fund account
model FundHistory {
id BigInt @id @default(autoincrement())
createdAt DateTime? @default(now())
status Json
nav Decimal
fundId String @map("fund_id")
strategyId BigInt @map("strategy_id") @default(0)

fund Fund @relation(fields: [fundId], references: [id])
}
// Tracks the history for a Fund account
model FundHistory {
id BigInt @id @default(autoincrement())
createdAt DateTime? @default(now())
status Json
nav Decimal
fundId String @map("fund_id")
strategyId BigInt @map("strategy_id") @default(0)

fund Fund @relation(fields: [fundId], references: [id])
}
I'm getting a column ""createdAt"" does not exist What am I missing??
No description
17 Replies
fascinating-indigo
fascinating-indigo•2y ago
Can you dump the schema of your database and see if your table even exists with the columns you expect
deep-jade
deep-jadeOP•2y ago
@Tristan Partin I checked, it does and the case is consistant with schema definition @Tristan Partin If you create a table following the schema mentioned above - are your able to make it a hypertable?
fascinating-indigo
fascinating-indigo•2y ago
@zatmonkey looking at it again, what happens if you remove the inner double quotes? I have never used timescale
deep-jade
deep-jadeOP•2y ago
if i remove them on FundHistory, then it does not find the table, on created at it does not make a difference @Tristan Partin
fascinating-indigo
fascinating-indigo•2y ago
If you run this migration on a local postgres instance, does it work?
deep-jade
deep-jadeOP•2y ago
haven't tried this i'd need to spin one up i mean i was looking for answers, not step to investigate
harsh-harlequin
harsh-harlequin•2y ago
The right syntax is : "select create_hypertable('"FundHistory"','createdAt');" I took ownership on the case you raised this morning and was replying to you via email 🙂 Simply put, the create_hypertable function calls a c function which handle the table name and the column name differently I assume that you saw this article: https://www.timescale.com/forum/t/pass-case-sensitive-view-name-to-timescaledb-functions/1021 This information is however incorrect. If you look at the source code of the create_hypertable function (https://github.com/timescale/timescaledb/blob/main/sql/ddl_api.sql#L23-L38), you will notice that it pass the parameters to a c function named ts_hypertable_create (https://github.com/timescale/timescaledb/blob/main/src/hypertable.c#L1647), itself calling ts_hypertable_create_time_prev (https://github.com/timescale/timescaledb/blob/main/src/hypertable.c#L1587 ). At the contrary of what the timescale community post says, only the relation name is casted as REGCLASS. The column name is instead cast as ::NAME, which eventually translate into: Oid table_relid = PG_ARGISNULL(0) ? InvalidOid : PG_GETARG_OID(0); and Name open_dim_name = PG_ARGISNULL(1) ? NULL : PG_GETARG_NAME(1);` So, using '" "' (single quote - double quote) does work for relation name, but doesn't for column names. For columns names, it appears that a simple single quote is sufficient, even if the column name is case sensitive Here is a quick reproduction from one of my test projects to demonstrate the above:
select create_hypertable("FundHistory",'createdAt'); -- ERROR: column "FundHistory" does not exist (SQLSTATE 42703) select create_hypertable("FundHistory","createdAt"); -- ERROR: column "FundHistory" does not exist (SQLSTATE 42703) select create_hypertable('FundHistory',"createdAt"); -- ERROR: column "createdAt" does not exist (SQLSTATE 42703) select create_hypertable('FundHistory','createdAt'); -- ERROR: relation "fundhistory" does not exist (SQLSTATE 42P01) select create_hypertable('"FundHistory"',"createdAt"); -- ERROR: column "createdAt" does not exist (SQLSTATE 42703) select create_hypertable('"FundHistory"','"createdAt"'); -- ERROR: column ""createdAt"" does not exist (SQLSTATE 42703) select create_hypertable('"FundHistory"','"createdAt"'::NAME); -- ERROR: column ""createdAt"" does not exist (SQLSTATE 42703) select create_hypertable('"FundHistory"','createdAt'); -- (3,public,FundHistory,t)
Additionally, please note that due to licensing restrictions, it's not possible to use compression with the TimescaleDB extension in Neon. Timescale provides this extension with two different licences: The Apache 2 Edition and the Community Edition. The first one is allowed for commercial uses, while the second is not. Unfortunately, the Apache 2 Edition doesn't include features provided by the Community Edition. See: https://docs.timescale.com/about/latest/timescaledb-editions/
xenophobic-harlequin
xenophobic-harlequin•2y ago
Not everyone in here is an expert on every technology. Please keep it respectful. Thanks Yanic and Tristan!
fascinating-indigo
fascinating-indigo•2y ago
@Yanic great investigation! Thanks for confirming my suspicion
deep-jade
deep-jadeOP•2y ago
sorry @Tristan Partin i wasn't trying to be disrespectful - somtimes chats come out dry @Yanic thanks for the email - super helpful
harsh-harlequin
harsh-harlequin•2y ago
With a real pleasure @zatmonkey !
deep-jade
deep-jadeOP•2y ago
(and impressed by the deep dig :D)
harsh-harlequin
harsh-harlequin•2y ago
Honestly, this seemed as weird to me that it did for you! I was really curious to get to the bottom of it, because it doesn't make much sense when you first hit the issue :p
deep-jade
deep-jadeOP•2y ago
alright, there were some additional secondary issues but I finally got it working thanks @Yanic
I'll assume compression is not critical for now. I'll think to add an aggregation table at some point I guess Hopefully you'll offer compression as an option in the future - looking more into it that's were the biggest value lies in this extension
fascinating-indigo
fascinating-indigo•2y ago
The license of the timescale extension prohibits that. You might be able to bring your own extension though. You would have to inquire about that The other option is using neon and timescale together
deep-jade
deep-jadeOP•2y ago
or y'all can offer it as an add on and rev-share with timescale
fascinating-indigo
fascinating-indigo•2y ago
right, I'll provide your feedback internally

Did you find this page helpful?