Селен Python пропускает необходимые элементы

ПОЖАЛУЙСТА, НЕ СКАЗЫВАЙТЕ, ЭТО ВОПРОС РАЗЛИЧЕН ОТ ПРЕДЫДУЩИХ ОДИН, IM ИСПОЛЬЗОВАНИЕ РАЗЛИЧНОЙ ЛОГИКИ ЗДЕСЬ

Я пытаюсь перебрать все отзывы пользователей (класс «part_entry») с этой страницы https://www.tripadvisor.com/Airline_Review-d8729164-Reviews-Cheap-Flights-or560-TAP-Portugal#REVIEWS

Если есть не английский комментарий, я хочу напечатать переведенную английскую версию. В противном случае, если комментарий уже на английском языке, я хочу напечатать сам английский. Но его код пропускает эти комментарии (а не печатать их). Также вы можете видеть на выходе, что комментарии печатаются дважды.

На этой странице есть 10 отзывов / комментариев (переведенных или не переведенных), и они должны печатать их все.

from selenium import webdriver from selenium.webdriver.common.by import By import time from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC driver = webdriver.Chrome() driver.maximize_window() url="https://www.tripadvisor.com/Airline_Review-d8729164-Reviews-Cheap-Flights-or560-TAP-Portugal#REVIEWS" driver.get(url) ctr=0 def expand_reviews(driver): # TRYING TO EXPAND REVIEWS (& CLOSE A POPUP) try: driver.find_element_by_class_name("moreLink").click() except: print "err" try: driver.find_element_by_class_name("ui_close_x").click() except: print "err2" try: driver.find_element_by_class_name("moreLink").click() except: print "err3" # FIRST EXPAND THE REVIEWS BY CLICKING "MORE" BUTTON expand_reviews(driver) for j in driver.find_elements_by_xpath("//div[@class='wrap']"): # FIND ALL REVIEW ELEMENTS for ent in j.find_elements_by_xpath('.//p[@class="partial_entry"]'): # FIND REVIEW TEXT # FIRST CHECK IF TRANSLATION IS AVAILABLE (IE NON ENGLISH COMMENTS) if j.find_elements_by_css_selector('#REVIEWS .googleTranslation>.link'): #print 'NOW PRINTING TRANSLATED COMMENTS' gt= driver.find_elements(By.CSS_SELECTOR,"#REVIEWS .googleTranslation>.link") size=len(gt) while (ctr<size): for i in gt: try: if not i.is_displayed(): continue driver.execute_script("arguments[0].click()",i) wait = WebDriverWait(driver, 10) wait.until(EC.element_to_be_clickable((By.XPATH, ".//span[@class = 'ui_overlay ui_modal ']//div[@class='entry']"))) com= driver.find_element_by_xpath(".//span[@class = 'ui_overlay ui_modal ']//div[@class='entry']") print com.text print "++" * 60 time.sleep(5) driver.find_element_by_class_name("ui_close_x").click() time.sleep(5) #loop+=1 except Exception as e: print "skipped" pass ctr+=1 # COMMENT ALREADY IN ENGLISH, PRINT AS IT IS else: print ent print "="*60 driver.quit() 

=============================== ВЫХОД ===================================================================== =========

 <selenium.webdriver.remote.webelement.WebElement (session="15b6c83088a289e59c544a2c7787d27d", element="0.40753995907133644-28")> ============================================================ <selenium.webdriver.remote.webelement.WebElement (session="15b6c83088a289e59c544a2c7787d27d", element="0.40753995907133644-29")> ============================================================ <selenium.webdriver.remote.webelement.WebElement (session="15b6c83088a289e59c544a2c7787d27d", element="0.40753995907133644-30")> ============================================================ <selenium.webdriver.remote.webelement.WebElement (session="15b6c83088a289e59c544a2c7787d27d", element="0.40753995907133644-31")> ============================================================ <selenium.webdriver.remote.webelement.WebElement (session="15b6c83088a289e59c544a2c7787d27d", element="0.40753995907133644-32")> ============================================================ On my change my flight without asking my opinion or offer another solution without paying extra I stay more than 10 hours in boarding of room I have the urge to have something to eat I haven not even able to rest after my flight c is inadmissible night I no longer would resume this company and would not advise a person to take ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ A little apprehensive before but quickly lifted. Very welcome and good service from the PNC, hot meal and good even for this short flight (1h50). Good punctuality and boarding more efficient ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Everything normal. Aircraft clean and almost full. Embarking on time, regular. Arrive slightly earlier. friendly and courteous staff. On board it was given a snack. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ In the recent past I have traveled a few times from Venice to Lisbon and from Venice to Oporto via Lisbon. Good facilities on land and aboard; friendly service, clean air, punctuality and competitive rates. recommended ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Sympathy and competence. The company strives to make passengers as comfortable as possible. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ On my change my flight without asking my opinion or offer another solution without paying extra I stay more than 10 hours in boarding of room I have the urge to have something to eat I haven not even able to rest after my flight c is inadmissible night I no longer would resume this company and would not advise a person to take ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ A little apprehensive before but quickly lifted. Very welcome and good service from the PNC, hot meal and good even for this short flight (1h50). Good punctuality and boarding more efficient ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Everything normal. Aircraft clean and almost full. Embarking on time, regular. Arrive slightly earlier. friendly and courteous staff. On board it was given a snack. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 

One Solution collect form web for “Селен Python пропускает необходимые элементы”

Один совет для удаления пути хромирования в каждом скрипте. Запустите chromedriver.exe в C: \ Python27 \ Scripts, чем вам не нужно помещать путь хромированной передачи в каждый скрипт, чем просто использовать driver = webdriver.Chrome()

Я запускаю этот код:

 from selenium import webdriver from selenium.webdriver.common.by import By import time from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC driver = webdriver.Chrome() driver.maximize_window() url="https://www.tripadvisor.com/Airline_Review-d8729164-Reviews-Cheap-Flights-or560-TAP-Portugal#REVIEWS" driver.get(url) ctr=0 for j in driver.find_elements_by_xpath("//div[@class='wrap']"): # FIND ALL REVIEW ELEMENTS for ent in j.find_elements_by_xpath('.//p[@class="partial_entry"]'): # FIND REVIEW TEXT # FIRST CHECK IF TRANSLATION IS AVAILABLE (IE NON ENGLISH COMMENTS) if j.find_elements_by_css_selector('#REVIEWS .googleTranslation>.link'): #print 'NOW PRINTING TRANSLATED COMMENTS' gt= driver.find_elements(By.CSS_SELECTOR,"#REVIEWS .googleTranslation>.link") size=len(gt) while (ctr<size): for i in gt: try: if not i.is_displayed(): continue driver.execute_script("arguments[0].click()",i) wait = WebDriverWait(driver, 10) wait.until(EC.element_to_be_clickable((By.XPATH, ".//span[@class = 'ui_overlay ui_modal ']//div[@class='entry']"))) com= driver.find_element_by_xpath(".//span[@class = 'ui_overlay ui_modal ']//div[@class='entry']") print com.text print "++" * 60 time.sleep(5) driver.find_element_by_class_name("ui_close_x").click() time.sleep(5) #loop+=1 except Exception as e: print "skipped" pass ctr+=1 # COMMENT ALREADY IN ENGLISH, PRINT AS IT IS else: print ent.text print "="*60 driver.quit() 

вывод, который я получаю:

 Quite comfortable for the economy class, with a friendly staff and good service. The food is good but could still be better. ============================================================ The pilot was amazing, soft take off, soft landing (even with ruf weather), very nice staff with amazing portuguese food and wine. The only downsize was the interior condition, although clean and without scratches or so you could see that is already aged. Appart from that all was good. ============================================================ Speedy check in process was very accurate and precise. They allowed cabin to be booked into the hold with no additional charges. Boarding was efficient and timely. the seats were very comfortable. Wide enough to fit me fairly comfortably with armrests that were able to lift during the flight. The really stand out thing for me was the leg space.... ============================================================ My country's flag airline, It has struggle to survive in a hard economic cycle. Clever choice of unique African and south american cities, guarantied its continuity.~ Do not expect a exquisite food, alcoholic drinks, down to beer and wine, forget white spirits. Good safety record. Pilots well trained, good maintenance. I have flight TAP for the last 40 odd years... ============================================================ Our first trip to Europe on a long flight both ways. The flight TO Rome was good. I am tall and have back issues, and thank God we were able to get exit row seats. This made all the difference in the world. The food served was fair to good. There were movies offered which helped pass the time and... ============================================================ On my change my flight without asking my opinion or offer another solution without paying extra I stay more than 10 hours in boarding of room I have the urge to have something to eat I haven not even able to rest after my flight c is inadmissible night I no longer would resume this company and would not advise a person to take ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ A little apprehensive before but quickly lifted. Very welcome and good service from the PNC, hot meal and good even for this short flight (1h50). Good punctuality and boarding more efficient ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Everything normal. Aircraft clean and almost full. Embarking on time, regular. Arrive slightly earlier. friendly and courteous staff. On board it was given a snack. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ In the recent past I have traveled a few times from Venice to Lisbon and from Venice to Oporto via Lisbon. Good facilities on land and aboard; friendly service, clean air, punctuality and competitive rates. recommended ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Sympathy and competence. The company strives to make passengers as comfortable as possible. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 

Обновление для обработки вещей в комментарии:

  1. Сначала установите ntkl, используя команду «pip install nltk»
  2. После завершения установки
  3. Открыть оболочку python ..ie, Idle
  4. Команда типа: import nltk
  5. теперь введите: nltk.download ()
  6. Пользовательский интерфейс откроется: нажмите «Модели» … найдите «punkt» и нажмите «Загрузить».
  7. После этого … нажмите на Corpora …. искать по стоп-словам и нажимать на скачивание

Одна из них выполняется: выполните следующую программу:

 from selenium.webdriver.common.by import By import time from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from nltk import word_tokenize from nltk.corpus import stopwords def detect_lang(text): lang_ratios = {} tokens = word_tokenize(text) words = [word.lower() for word in tokens] for language in stopwords.fileids(): stopwords_set = set(stopwords.words(language)) words_set = set(words) common_elements = words_set.intersection(stopwords_set) lang_ratios[language] = len(common_elements) return max(lang_ratios, key=lang_ratios.get) driver = webdriver.Chrome() driver.maximize_window() url="https://www.tripadvisor.com/Airline_Review-d8729164-Reviews-Cheap-Flights-or570-TAP-Portugal#REVIEWS" driver.get(url) ctr=0 time.sleep(5) def expand_reviews(driver): # TRYING TO EXPAND REVIEWS (& CLOSE A POPUP) try: driver.find_element_by_class_name("moreLink").click() except: print "err" try: driver.find_element_by_class_name("ui_close_x").click() except: print "err2" try: driver.find_element_by_class_name("moreLink").click() except: print "err3" # # FIRST EXPAND THE REVIEWS BY CLICKING "MORE" BUTTON expand_reviews(driver) time.sleep(10) for ent in driver.find_elements_by_xpath('.//div[@class="entry"]/p[1]'): # FIND REVIEW TEXT lang = detect_lang(ent.text) if (lang == 'english'): print ent.text print "=="*30 else: if driver.find_elements_by_css_selector('#REVIEWS .googleTranslation>.link'): gt= driver.find_elements(By.CSS_SELECTOR,"#REVIEWS .googleTranslation>.link") size=len(gt) while (ctr<(size/2)): for i in gt: try: if not i.is_displayed(): continue driver.execute_script("arguments[0].click()",i) wait = WebDriverWait(driver, 10) wait.until(EC.element_to_be_clickable((By.XPATH, ".//span[@class = 'ui_overlay ui_modal ']//div[@class='entry']"))) com= driver.find_element_by_xpath(".//span[@class = 'ui_overlay ui_modal ']//div[@class='entry']") print com.text print "++" * 60 time.sleep(5) driver.find_element_by_class_name("ui_close_x").click() time.sleep(5) #loop+=1 except Exception as e: print "skipped" pass ctr+=1 

Это напечатает следующий результат:

  Speedy check in process was very accurate and precise. They allowed cabin to be booked into the... read more ============================================================ Very pleasant flight, excellent service on board and on the ground, the best seats in the Buisness Class and Top Food and drinks during the flight. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Perfect atendimento.Bom care of Commissioners and Commissioners, punctuality. Good movies offered. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Overall, a good flight! Time (departure and arrival). Enough time for the change to Lisbon. Very nice crew! ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ It was a flight noturno.Teve strong turbulence and I could not dormir.Rezei all night. After all it was a decent trip. only regret the discomfort of the aircraft but praise the good atendimento.Toda the crew was very kind and helpful.The journey back was quieter. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ The food really is not the best to tell the truth, I could not even eat. But the service is very good. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Kindness, professionalism, and willingness on the part of the crew: good landing and includes drinks and light dinner ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ On 9 October flown with this company. By delayed entering the unit departed late. Atmosphere Loos routine operation. The evening meal consisted of a tuna sandwich and a liquid plum in plastic vial. A choice of meat or cheese was not there. For me and many others so no meal on this flight. Downright depressing. ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Speedy check in process was very accurate and precise. They allowed cabin to be booked into the hold with no additional charges. Boarding was efficient and timely. the seats were very comfortable. Wide enough to fit me fairly comfortably with armrests that were able to lift during the flight. The really stand out thing for me was the leg space. I'm 6ft 4in and I was able to fully extend my legs under the chair in front of me. if for this reason alone I have started looking for other destinations they fly to. In flight entertainment was none existent but then it was only a short haul flight so I won't fault them. Food snack and beverages were included in the price. The in flight attendants were professional, courteous and well presented. I will definitely use them again in the future! ============================================================ My country's flag airline, It has struggle to survive in a hard economic cycle. Clever choice of unique African and south american cities, guarantied its continuity.~ Do not expect a exquisite food, alcoholic drinks, down to beer and wine, forget white spirits. Good safety record. Pilots well trained, good maintenance. I have flight TAP for the last 40 odd years I have seen many faces and crisis, however customer service really Bad. ============================================================ Our first trip to Europe on a long flight both ways. The flight TO Rome was good. I am tall and have back issues, and thank God we were able to get exit row seats. This made all the difference in the world. The food served was fair to good. There were movies offered which helped pass the time and blankets because the cabin got very cold. We had to sit in Lisbon Airport for six hours to complete our journey. It was tiring but still the trip was good (fun even) The flight back home was not as pleasant. The first leg from Rome to Lisbon was horrible. The woman in front of me kept slamming her seat up against my legs and telling me she had to lay back and to put my tray table up. The flight attendant SAW this happen and did nothing. I found the crew to be very unfriendly on all legs of the flight. They were not warm and friendly and even bordered on rude at some point. I would probably fly this airline again because the price was right and it was acceptable.... ============================================================ 
  • Как очистить Instagram с помощью BeautifulSoup
  • Проблема скремблирования созданного javascript контента с помощью Selenium и python
  • Получение n-го элемента с помощью BeautifulSoup
  • Данные веб-соскабливания с интерактивной диаграммы
  • Python переадресовывает, а затем загружает страницу?
  • Python - проблема с акцентированными символами при очистке данных с веб-сайта
  • Может ли телеграмм бот читать сообщения канала
  • Извлечение текста без тэгов HTML с помощью Beautifulsoup Python
  • Python - лучший язык программирования в мире.