The same news headlines from the site are repeated many times

Question

The same news headlines from the site are repeated many times

I'm a beginner.I found out about such a thing as BeautifulSoup a couple of days ago.So don't judge me harshly if I made a stupid mistake.I wanted to parse the headlines of the main news with championat.com .In the end,it turned out, but with one problem(see the question title).Language version 3.9

responce=requests.get(championat_url, headers=headers)
            soup=BeautifulSoup(responce.content, 'html.parser')
            items=soup.findAll('div',class_='news-item__content')
            comps=[]

            for item in items:
                    comps.append({ 'title': item.find('a',class_='news-item__title').get_text(strip=True)})
                                                      
                    for comp in comps:
                            print(comps['title'])

As a result:

0

python-3.x requests bs4 web-scraping

Author: Наруто Узумаки, 2020-12-03

Source

1 answers

score 2 · Accepted Answer

    for item in items:
            comps.append({ 'title': item.find('a',class_='news-item__title').get_text(strip=True)})
                                              
            for comp in comps: # <- тут лишний отступ
                    print(comps['title']) # <- тут неправильное название переменной

You have an extra indent in the first place, so you have a loop in the loop. And secondly, you do not print the loop variable in the second loop, but the entire list each time. It would probably be correct to do so:

    for item in items:
            comps.append({ 'title': item.find('a',class_='news-item__title').get_text(strip=True)})
                                              
    for comp in comps: 
            print(comp['title'])