Python script that extracts Wikipedia pages and records them to two XML files

There are two Wikipedia category pages

1) [login to view URL]:All_NPOV_disputes

2) [login to view URL]:Good_articles/all

I need a python script that will

1) extract ALL the Wikipedia pages linked to from the 1st page (in the "Pages in Category "All NPOV Disputes" section) and

2) extract RANDOM 5000 (default setting) Wikipedia pages linked to from the 2nd page ("good articles" — from randomly chosen categories),


Convert them to the two XML files where

a) one file contains the actual articles (with an id starting from 0000000 to 0006000), the url, and the full text — like in the example upload articles-trained-byarticle.

b) the other file contains the id, the url, and the npov score, which equals NPOV = true for the articles imported from Category:All_NPOV_disputes and NPOV = false for the articles imported from Wikipedia:Good_articles/all

The script should have additional settings (initialized in the jupyter notebook when calling the script) that

1) can specify the range of the size of the text to be imported (e.g. default 0 to 10000 Kb)

2) can specify the type of articles to be imported (an array of Wikipedia page categories accepted, e.g. "Biographies", default = all)

3) can specify which source to use for NPOV = true and which source to use for NPOV = false (default settings - above)

4) can specify how many pages to be imported from each page(default: 5000, 5000)

note: the NPOV page is paginated, so you'll have to take this into account

The script should run in a Jupyter Notebook and have clear instructions for installing all the dependencies through anaconda or pip.


1) The script as above with all the settings

2) The processed dataset with the default settings above (that is, 2 XML files with extracted articles and NPOV score)

Taidot: tiedonlouhinta, Python, Tietojen kaavinta verkosta, Wikipedia

Näytä lisää: wikipedia cirrus dump, wikipedia api, tamil wikipedia dump, python extract text from wikipedia, wikipedia to text converter, wikipedia corpus python, wikipedia xml dump example, python parse wikipedia dump, script xml files, python script text files, script search xml files folder string, mysql script parse xml files linux, php script uploading xml files mysql, script modify xml files, reading parsing xml files action script, create script modifies xml files, python script convert xml csv, python library parse wikipedia xml, python script parse csv files, script will search extract strings xml files

Tietoa työnantajasta:
( 7 arvostelua ) Berlin, France

Projektin tunnus: #19257124

19 freelanceria on tarjonnut keskimäärin %project_bid_stats_avg_sub_26% %project_currencyDetails_sign_sub_27% tähän työhön


Hello there. Just read your job description and I am very interested in it. As a scrap expert, I can help you well. As you can see my profile, I have many good experiences in scraping with python. You can achieve y Lisää

€400 EUR 3 päivässä
(92 arvostelua)

Hi there,I am Python Web Scraping expert from Bosnia & Herzegovina,Europe. I have carefully gone through with your requirements and I would like to help you with this project ! I can start immediately and finish it wi Lisää

€155 EUR 3 päivässä
(89 arvostelua)

Hi, I hope you're having a wonderful day i have done scrapping almost on Half of Worldwide web including eCommerce giants (Amazon, eBay, craigslist) News Feed, Social media websites, API's. I develop my own tools Lisää

€125 EUR 7 päivässä
(24 arvostelua)

Hello, I have gone through your job posting and become very much interested to work with you. I am an expert in this field. I have already completed several projects like this. For evidence you can see my profile. Lisää

€250 EUR 5 päivässä
(40 arvostelua)

Hi. I saw your description carefully and I think I can help you. I'm Web scraping expert, I have more experience in python web scraping. if you want to know more imformation about me ,please see my profile. Scrapin Lisää

€100 EUR 3 päivässä
(33 arvostelua)

Hi, I am experienced on Python, XML and web scraping/bot programming, I check your project's details very carefully, I can complete your work 100% perfectly and I can give you a perfect scraper to scrape data perfec Lisää

€100 EUR 12 päivässä
(37 arvostelua)

Hi, employer. I am strong in python scrapping and automation I've read your proposals carefully and I think I can do it. I have many previous works in this work and I will complete your project definitely. The pe Lisää

€155 EUR 3 päivässä
(16 arvostelua)

Bonjour ! I can make you Python script that will extract wiki pages into xml files according to your requirements. If interested - I can make you a sample output files, so you can be sure that I am able to do that job.

€166 EUR 7 päivässä
(24 arvostelua)

‌Hi, I have gone through your requirement to scrape lots of websites. I am EXPERT in building scraping tools /scripts. Hence, I can SURELY work on your project. I am having 4 YEARS of EXPERIENCE in developing PHP-PYTHO Lisää

€222 EUR 3 päivässä
(44 arvostelua)

Hello there! Ill use python for this task. I would like to talk more about the project through chat. please have a look at my reviews and ping me! My skills & experience: -- 2.8 years of experience in building Lisää

€230 EUR 6 päivässä
(35 arvostelua)

Hello, i'm an experienced Python programmer and also a fan of Jupyter Notebooks. I already did some projects here on freelancer with them. For the crawling task you described i'd propose using the Python crawling l Lisää

€160 EUR 3 päivässä
(14 arvostelua)

Greetings! I hope you are doing great. I am highly professional in managing script writing projects. Please contact so I may assist you. Samples available upon request. Thank You, Revival

€250 EUR 5 päivässä
(9 arvostelua)

Hello! I have briefly read the description on python-script-that-extracts-wikipedia development project, and I can deliver as per the requirements however I need us to discuss for more clarity on the details, Lisää

€250 EUR 3 päivässä
(6 arvostelua)

Hello I am a python developer with experience scrapping data from wikipedia with beautiful soup, I can do this in a week for 200 eur, talk to me in chat for more details.

€200 EUR 7 päivässä
(6 arvostelua)

I Will Do Data Entry,Data Analysis,Data Mining,Internet Research I specialize in : ? Offline and Online Data Entry ? Data Mining ? Data Analysis ? Copy Paste Task ? Data Capturing From Any Website ? G Lisää

€250 EUR 3 päivässä
(2 arvostelua)

Hello, after reading your project details we believe we are suitable for this project. We are a Python Developers with 5+ years experience in php scripts. your project is very interesting for us and we have confidence Lisää

€250 EUR 3 päivässä
(25 arvostelua)

Dear employer, Hi I have done my M.Sc. thesis using Python and MATLAB. It was about developing a numerical model for simulating fluids flow through porous media. I developed the main code in Python and developed Lisää

€120 EUR 3 päivässä
(5 arvostelua)

✅Hello, Nice to meet you. I hope to work with you. Experience fields: - Php(Laravel, codeigniter) - Java(Struts, Spring Framework) - Python(Django, Selenium, Scrapping) - Mobile(Android, iPhone, IOS, iPad) - No Lisää

€100 EUR 3 päivässä
(3 arvostelua)

⭐⭐⭐⭐⭐⭐⭐⭐⭐⭐ Hi I read your job description carefully and I can do your job perfectly. I have developed many websites So I can know what you mean and I am ready for you now. If you hire me, I will finish your job A Lisää

€155 EUR 3 päivässä
(0 arvostelua)