Just about done with my Twitter scraper
Just about done with my Twitter scraper. The experience was alright, but I ended up building 680 times in the process of completing it 😅 . The main problem I faced was the ide was difficult to work with without linting etc, and I didn't want to run it locally since I thought it would behave differently without the input setup etc. Maybe you can do it all locally, but I just found the experience with the ide very frustrating without linting. I think it could be a great feature to add.
10 Replies
absent-sapphire•2y ago
Hey @Jonathan Larson, thank you so much for feedback. It's quite determination to build it so many times 😄
Can you please let me know a little bit more details? What template you used, and what other things you would imagine in the web IDE outside of linting? The local env should work the same as web IDE and the input would be defined in input_schema.json and you could make the changes there.
If you give me a bit more details on the issues on local env, it would be great 🙂
We're now fixing a bug for the linting and it should work after that in web IDE. This is amazing feedback.
@Jonathan Larson just advanced to level 1! Thanks for your contributions! 🎉
frail-apricot•2y ago
So you're saying I can just use input_schema.json as my input by using default values, or do you create another file?
In terms of the web IDE linting is the main thing for sure. There's things like github copilot that I haven't looked into but am assuming you guys couldn't do because of licensing issues etc.
One thing I noticed with the web IDE while working in it is that the page would seem to occasionally update, whether I had auto save on or not, and then it would jump my cursor to the bottom line. This made things a bit more difficult since it usually happened while I was typing and i had to scroll back up to where my code was.
It would also be great if the web IDE allowed resizing for the different elements, since as is there's no way hide the minimap or explorer. In addition, making the only way to enlarge the code window fullscreening means that even though I can have to windows working side by side with Mac, I can't quickly switch tabs on the window with apify, (map resizes dynamically when tab is changed, so that's good though)
absent-sapphire•2y ago
@Jonathan Larson Yes, I would use the input_schema and use prefill with some values. I don't think you need any other file. @Lukas Krivka can correct me if I'm wrong.
Thanks for the rest of the feedback, I added it to our improvements for future iterations for web IDE and overall developer experience :)) It all makes sense.
Just one more question though - would you develop Actors in web IDE if you would know the input works normally in local env as well?
frail-apricot•2y ago
Hmm, I would say that local env is still better since my hardware is much better than the compute offered for building (and it’s probably my fault for building so much)
One other thing I forgot is version control. It seems I can reuse old builds, but I didn’t see a place to actually get my old code. (Not sure if that’s not a thing or I missed it)
I will say that the platform makes sense while working on the json files since you can quickly see a preview, and testing different inputs through the interface due to its increased simplicity, but for everything else local is still probably the way to go.
I understand your dilemma, since getting people to build (and importantly deploy) on Apify makes you a profit with the compute up-charge vs letting developers build everything locally, likely encouraging them to deploy on their own system instead.
I think maybe just improving the web IDE a lot is a start, along with great templates that work specifically for it. Unfortunately I was happy but surprised the Actors even work offline, since I had figured Apify would want to ensure their platform is used for compute.
@Jonathan Larson just advanced to level 2! Thanks for your contributions! 🎉
absent-sapphire•2y ago
@Jonathan Larson Sorry for not replying earlier. I had the message but forgot to send it.
Yeah, we actually want users to try web IDE to quickly see how our platform works and how fast you can create actors but then continue developing locally. The input & output schema is powerful if you plan to publish actors to the Store.
We'll definitely think of ways to work on it and improve it and I really appreciate your time and feedback. If you have anything else, always push it to the #💫feature-request channel :))
absent-sapphire•2y ago
@Jonathan Larson Regarding version control - you can fork the code in the web IDE and basically choose from the drop-down the version you want to edit. That version should include the old code.
There's currently no way how to get specific old code from previous builds though :/
foreign-sapphire•2y ago
I strongly recommend building anything complex locally. The Apify SDK works pretty much 1:1 locally vs platform
rare-sapphire•2y ago
@Jonathan Larson you can create an INPUT.json file on the
storage/key_value_stores/default folder on your local