rising-crimson

Twitter scraper again - user name

I would like to scrape only the user name that wrote the tweet and not all of the user info. the end result should be username and tweet. How would I go about doing that? Thank you

5 Replies

MEE6•3y ago

@Ronablu just advanced to level 1! Thanks for your contributions! 🎉

compatible-crimson•3y ago

Hi @Ronablu, while exporting the dataset, you can select what fields you want to select/omit.

rising-crimsonOP•3y ago

Hi @vojtechmaslan , the scraper doesn't collect the user name if the add user info is not checked. I don't need all of the user data. I just need the user name. It makes the file larger and with data I have no use for, and consume more CU. is there a way to collect just the user name?

compatible-crimson•3y ago

The Twitter scraper allows you to transform the output before pushing it to the dataset using the extendOutputFunction input. Something like:

async ({ data, item, page, request, customData, Apify }) => {
  return {
    screen_name: item.user.screen_name,
    full_text: item.full_text,
  };
}

async ({ data, item, page, request, customData, Apify }) => {
  return {
    screen_name: item.user.screen_name,
    full_text: item.full_text,
  };
}

However, this will not have an effect on the CU consumption.

rising-crimsonOP•3y ago

Hi @vojtechmaslan Sorry for the late reply. I'm looking for a solution in the Extend Scraper Function. To just add a username column. Now in order to get that information I use an Excel formula to extract the username from the URL. Is there a way for me to implement that into the Extend Scraper Function? Thank you

Gaming

Programming

Twitter scraper again - user name

Did you find this page helpful?