Writing data from Kafka topic to database
Hello,
I am trying to create a simple replication of DML and DDL operations from a source database to a target database. The hosts are running Almalinux 9 and PostgreSQL 16.
I am using pgstream to send data changes to a Kafka topic. I then try to fetch these via Kafka Connect with the JDBC Sink Connector and apply them to the target database.
Unfortunately, there are problems with the Sink Connector. For example, whenever I create a new test table using CREATE TABLE, I get the error message from the attached file.
To me it looks like some sort of format error...
Can anyone explain what I am doing wrong here? Has anyone ever successfully implemented such a project? Perhaps with alternative software solutions?
Thanks for any advice.
EDIT: Seems like the JDBC Sink Connector does not support DDL operations
Any alternatives out there?
I am trying to create a simple replication of DML and DDL operations from a source database to a target database. The hosts are running Almalinux 9 and PostgreSQL 16.
I am using pgstream to send data changes to a Kafka topic. I then try to fetch these via Kafka Connect with the JDBC Sink Connector and apply them to the target database.
Unfortunately, there are problems with the Sink Connector. For example, whenever I create a new test table using CREATE TABLE, I get the error message from the attached file.
To me it looks like some sort of format error...
Can anyone explain what I am doing wrong here? Has anyone ever successfully implemented such a project? Perhaps with alternative software solutions?
Thanks for any advice.
EDIT: Seems like the JDBC Sink Connector does not support DDL operations
Any alternatives out there?error.txt5.83KB
