Dynamic Web Scraping with Selenium Part- VI- Instagram
This time I was messing with Instagram and the task was to send DMs to a select users. Everything was very easy. The only challenge I faced was to select the user for DM. Let me show what it was.
When we click on the ‘send message’ button a dialogue box appears. In the input we have to write the username of 1 or more persons and then select the user from the list shown below the input.
In our case we have exact usernames. So, we can deduce that the first user shown in the list, highlighted as ‘2’, is the user which is our target for DM. So, we inspect this element and find that it’s a ‘div’ element with ‘role=button’. This has been highlighted as ‘3’ in the image.
We setup the selenium script to find this element in the dialogue box and click this element whose role is button and text is same as the username. What happened is that the script did click the element and proceeded but actually the action was not performed on the screen. I mean the element was clicked successfully but nothing happened in action.
After careful observations I found out that actually this element was not the one which gets the click event. Highlighted as ‘4’ is the last ‘div’ element inside our target element. This element was being triggered whenever the main user element was hovered or clicked.
So, how did I solve this problem?
I setup the script to move to the main element, highlighted as ‘3’ and then click the ‘4’ element rather than clicking the ‘3’ element. And, it worked.
user_main_element = dialogue_element.find_elements(By.XPATH, "//div[@role='button']")[0]
ActionChains(self.driver).move_to_element(user_main_element).perform()
time.sleep(5)
inner_btn = self.driver.find_element(By.XPATH, "/html/body/div[2]/div/div/div[3]/div/div/div[1]/div/div[2]/div/div/div/div/div/div/div[3]/div/div/div[1]/div[2]")
ActionChains(self.driver).click(inner_btn).perform()A very important point in each profession is that we learn by experience. Without experiment we can not learn a lot and can not get better and better result. And, learning from the others’ experience is also a genius.
So, don’t forget to subscribe so that you may learn from my experience and apply on your experience to nurture great ideas. Also, don’t forget to give a clap of appreciation so that I may continue sharing my experience.
See you next time,