当前位置:网站首页>Can I write a crawler without writing code? The teacher said yes, one of the reptiles without coding.

Can I write a crawler without writing code? The teacher said yes, one of the reptiles without coding.

2020-12-07 19:10:00 Dream eraser

Today is the first day of continuous writing 16 / 100 God .
If you have ideas you want to communicate 、 technology , Feel free to leave a comment in the comments section .

A lot of people put technology in a higher position , But technology is really just a means to achieve a goal , No programming at all , Don't want to write code , Can we implement a crawler ? The answer, of course, is yes .

This series of articles will take you to the door of codeless writing crawler , Everyone can complete the crawler , Everyone can collect the data they need .

Relieve visual fatigue

 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .

Web Scraper Introduction and installation

The definition given by the official website : Making web data extraction easy and accessible for everyone

This plug-in belongs to browser plug-in , The official website is : https://www.webscraper.io/, The download address in the official website is shown in the figure .

 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .

The plug-in is based on Google browser and fox browser , But visiting the Google plug-in store requires scientific access to the Internet , It's not easy to do , It is suggested to search and download directly in the search engine , If you can't retrieve , You can use the address I found for you .

#  Data source network , You can look for it yourself , There are a lot of resources .
https://pan.baidu.com/s/1CfAWf0wMO6WqicoUgdYgkg  Extraction code : nn2e

In the open page , Download folder and crx All the documents can be , There is no difference in installation .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .

Browser plug-in installation

Google browser plug-in installation directly in the browser address bar input chrome://extensions/, Open the page to open developer mode . Drag and drop crx File to the browser plug-in page .

It's easy to make mistakes in the way you drag and drop Invalid package , The error shown in the figure below .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
The solution is very simple , modify crx File suffix zip, After decompression , Install through folder . Pay attention to click Load the unzipped extender , Select the unzipped folder .

 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
Installation completed , Show the plug-in to the browser extension location . Follow the following two steps , Finally, a spider web icon appears in the browser to indicate success .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
If you don't know the installation process yet , You can refer to CSDN Development Assistant home page ,Web Scraper The installation process is consistent with the developer assistant .
Plug in website :https://plugin.csdn.net/

In addition to detecting plug-in icons , We also need to check whether the function exists in the developer tool . The developer tool adds a new item at the end Web Scraper, It's ok if this item appears .

 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .

Find a website to practice

Web Scraper Especially like a piece of software , Just touch found that no button will click , Use it and discover , The function is very simple , Just use the mouse to complete all the operations , Basically, you don't have to write code ( I still have to type a few keystrokes once in a while ).

Learn the skills of software operation , The best way is to learn from it , Use it and use it , Don't take in the details .

The first one to climb , Tentatively 【 Everyone is a product manager 】 This website .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
open Web Scraper after , In fact, it's easy to know what to do next , Because only the third of the three buttons can be clicked .

Click on Create new sitemap To create a website guide ,Sitemap name Understand it as the name of a reptile , Below Start URL Start page address , Can be for more than one URL, This case does not involve , The following article contains this content . We can put sitemap It's like a crawler , This makes it easier to understand .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .

Input sitemap name And start url after , You can create a website to be crawled . The input is as follows :

  • sitemap name : woshipm
  • start url : http://www.woshipm.com/

After the creation, the following interface will be entered by default , Pay attention to two positions , The first red box selection area ,_root Represents the root node , Just remember it's a noun , Indicates the structure of the page you just set . The button below is a selector , Explain later .

 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
Note at this time sitemaps You can click on , After clicking, it will list all the sitemap, In fact, you can write multiple crawlers , This place is a list .

 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .

When you click on a crawler to go inside , You'll find another detail change in the menu .

 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
After entering a specific crawler program , The drop-down list above can be operated , But for the time being .

Click on add new selector after , The jump page is shown in the figure below , There are two important places , The first is Type, The selection of this value affects the input content of the interface , There's another one that needs attention ,parent selectors Parent selector , Notice that the value inside is now _root.
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
The next operation is very interesting , Let's click on Select Button .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
Static graphs are not rarely viewed , Use dynamic diagrams to show , After that, click the button , You can go to the open page and choose , For example, the location I click on is the title of the document , After clicking on one , The background turns red , Click on the second , After this ,Web Scraper It will automatically select all the contents that conform to your click rule , That is to say, the pictures below are all in red . You can check if there is a problem , If there is no problem , Click... In the middle of the window Done selecting To end the selection .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
After finishing the selection, click the button and the final effect as shown in the figure .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
stay id After writing a name in Zhongshu , Save the selector .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
The jump page is shown in the following figure :
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
Click on the right Data preview after , The interface that appears shows only one piece of data , It's not consistent with the number of data we expect to get , Click on Edit Get into selector Continue to modify .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
After checking the button shown below , Continue to operate .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
Save selector , Back to the previous screen , Click on Data preview The occurrence of multiple data indicates that the operation is completed .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
The next step is to start the crawler , Operate as shown in the figure below .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
Click on Scrape To enter the interface, you need to set the number of requests and the request delay of each page , Generally, keep the default .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
Pop up a small browser interface , After that, it will jump to the following interface .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
Click the refresh button on the page , Get the data, complete the task .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
The last step is to export the data to local .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
Note that the column names in the final local file are as follows , The first two are automatically generated by the system , The last column is the data we want , And the column name is just defined id value .
 Can I write a crawler without writing code ? The teacher said that he could , One of the codeless reptiles .
Here we are , The first case has been completed . Complete a crawler without a single line of code .

Written in the back

I'm sure a friend will say when I study for the first time , What a bother , It's better to write a program , use requests It's easy to climb . It's not , As the follow-up courses unfold , You'll find it works well , On some websites , In particular, there are anti climbing means of the website , This is easy to use 、 Brutal 、 It works .

Want to learn Python Reptiles , You can subscribe to the eraser column ~ 《 A hundred examples of reptiles 》 Click to find the surprise


If you want to build a close relationship with a blogger , You can follow bloggers , Or focus on Blogger official account “ Non undergraduate programmer ”, Learn how a non undergraduate programmer grows up .
Blogger ID: Dream eraser , I hope you all. give the thumbs-up Comment on Collection .

版权声明
本文为[Dream eraser]所创,转载请带上原文链接,感谢
https://chowdera.com/2020/12/20201207190856282s.html