Parsing Stray Text With Scrapy

Any idea how to extract 'TEXT TO GRAB' from this piece of markup:

Solution 1:

Not ideal:

text_to_grab = response.xpath('//span[@class="navigation-pipe"]/following-sibling::text()[1]').extract_first()

Solution 2:

It's not an ideal solution but it should do the trick:

from scrapy import Selector

content="""
<spanclass="navigation_page"><span><aitemprop="url"href="http://www.example.com"><spanitemprop="title">LINK</span></a></span><spanclass="navigation-pipe">&gt;</span>
    TEXT TO GRAB
</span>
"""
sel = Selector(text=content)
item = sel.css(".navigation_page::text")
print(item.extract()[-1].strip())

OR like this:

sel = Selector(text=content)
item = ''.join([' '.join(items.split()) for items in sel.css("span.navigation_page::text").extract()])
print(item)

Output:

TEXTTO GRAB

Share

You may like these posts

• 
  Attributeerror: 'module' Object Has No Attribute 'asyncoreconnection'
• 
  How To Install Scrapy On Ubuntu 16.04?
• 
  Scrapy - Importing Excel .csv As Start_url
• 
  Csv Exports - Ordering Of Columns Using Scrapy Crawl -o Output.csv