Prompt: Write a python program that scraps table data from a webpage. Figure out an efficient and accurate way to retrieve table data from the web page which adds more data to the page when the viewer scrolls down the page using the button on the screen. The web page to use is https://www.tradingview.com/etf-screener/. From this page, check that there are multiple rows of data under columns labelled below the row that starts with Symbol, Price, Change %, and all other columns of this table. On the first page of the web page, there are about 100 rows of information.
Method to scrolling down this webpage to load additional data sets:
Once this data is confirmed present, about 100 lines, the web page has, on its right edge, after the column heading Focus and plus sign, a button that can be used to scroll down the web page. Grab this scroll down button, and while holding this button, move it all the way down to the bottom of the web page. By doing this, the web page loads additional 100 rows of data in the table. Confirm that after scrolling down, additional rows of data have been added to the web page. Keep scrolling down the web page, so that more rows of data are added to the page. Keep scrolling until no further lines are added to the table. Once the maximum lines are reached, copy all the rows of data into a csv file. Provide the count of all the rows that have been added to the csv file and the print the first 10 rows on the screen.
Pre-processing of Data before saving:
When data is to be copied into the csv file, pay attention to the column Symbol. In the entries of the rows, the information available is a few characters, and then there is name in the form of a weblink. Break up the symbol column into two columns. First column should have the characters and label it as symbol, and the second new column, call it name, should have the name stored in it that is like a weblink. Do not make any other changes to the data, only first column has to be split into two columns at the time of storage.
RESULT: Claude did very well for this prompt. Perplexity also did ok.
Bad performers were ChatGPT, Grok, Manus, and Losers were Gemini, Deepseek, Copilot, Qwen, Kimi, Meta AI, Mistral, Ernie, Minimax