Railway•10mo ago

I want to run Airbyte

I would like to use an ETL and I heard that Airbyte is good so I'm trying to set it up as a service from github but Im runnig into errors. Are there any other recommended ETL's that will work out of the box? I will post my errors and journey to set it up here.

6 Replies

Percy•10mo ago

Project ID: fd15b2e5-71af-4fe9-a6f1-fcc5ecb8357c

Peauts•10mo ago

fd15b2e5-71af-4fe9-a6f1-fcc5ecb8357c first set of build errors:

Dockerfile:19

-------------------

17 |     # build phase

18 |     COPY . /app/.

19 | >>> RUN --mount=type=cache,id=s/65b1d2a2-3ccd-4a0b-963b-a2122d0f0a77-/root/gradle,target=/root/.gradle ./gradlew build -x check

20 |

21 |

-------------------

ERROR: failed to solve: process "/bin/bash -ol pipefail -c ./gradlew build -x check" did not complete successfully: exit code: 1

 

Error: Docker build failed

Dockerfile:19

-------------------

17 |     # build phase

18 |     COPY . /app/.

19 | >>> RUN --mount=type=cache,id=s/65b1d2a2-3ccd-4a0b-963b-a2122d0f0a77-/root/gradle,target=/root/.gradle ./gradlew build -x check

20 |

21 |

-------------------

ERROR: failed to solve: process "/bin/bash -ol pipefail -c ./gradlew build -x check" did not complete successfully: exit code: 1

 

Error: Docker build failed

more context on this run:

#10 71.52

#10 71.52 FAILURE: Build failed with an exception.

#10 71.52

#10 71.52 * Where:

#10 71.52 Build file '/app/airbyte-cdk/python/build.gradle' line: 3

#10 71.52

#10 71.52 * What went wrong:

#10 71.52 An exception occurred applying plugin request [id: 'airbyte-docker']

#10 71.52 > Failed to apply plugin 'airbyte-docker'.

#10 71.52    > A problem occurred starting process 'command 'docker''

#10 71.52

#10 71.52 * Try:

#10 71.52 > Run with --stacktrace option to get the stack trace.

#10 71.52

#10 71.52 FAILURE: Build failed with an exception.

#10 71.52

#10 71.52 * Where:

#10 71.52 Build file '/app/airbyte-cdk/python/build.gradle' line: 3

#10 71.52

#10 71.52 * What went wrong:

#10 71.52 An exception occurred applying plugin request [id: 'airbyte-docker']

#10 71.52 > Failed to apply plugin 'airbyte-docker'.

#10 71.52    > A problem occurred starting process 'command 'docker''

#10 71.52

#10 71.52 * Try:

#10 71.52 > Run with --stacktrace option to get the stack trace.

Peauts•10mo ago

I found https://www.mage.ai/ in the list of templates I will try them. Would love to see Airbyte added since they seems like a great solution for etl!

Mage

Give your data team magical powers

Mage is an open-source data pipeline tool for transforming and integrating data.

Brody•10mo ago

I would attempt to add airbyte, but it's java

Peauts•10mo ago

Don’t knock Java. Solid language. Has a few skeletons in the closet but when you garnish it with spring its digestible.

Brody•10mo ago

I didn't knock it, I just don't know java lol

Gaming

Programming

I want to run Airbyte