全球主机交流论坛

标题: 请教会抓包的彦祖，怎么得到知乎瀑布分页规则火车头 [打印本页]

作者: z6045670 时间: 2020-8-28 22:39
标题: 请教会抓包的彦祖，怎么得到知乎瀑布分页规则火车头
我要采集的知乎问答
https://www.zhihu.com/search?type=content&q=mjj

自己按着抓包教程抓到的是下面这种虽然是所属关键词的内容但数据不对跟页面上的不一致

https://api.zhihu.com/search_v3?advert_count=0&correction=1&lc_idx=0&imit=20&offset=20&q=mjj&search_hash_id=3a30b7f93413a7ba8e9d7a4a886f83ed&show_all_topics=0&t=general&vertical_info=0%2C1%2C0%2C0%2C0%2C0%2C0%2C0%2C0%2C1

https://api.zhihu.com/search_v3?advert_count=0&correction=1&lc_idx=24&limit=20&offset=40&q=mjj&search_hash_id=3a30b7f93413a7ba8e9d7a4a886f83ed&show_all_topics=0&t=general&vertical_info=0%2C1%2C0%2C0%2C0%2C0%2C0%2C0%2C0%2C1

https://api.zhihu.com/search_v3?advert_count=0&correction=1&lc_idx=64&limit=20&offset=80&q=mjj&search_hash_id=3a30b7f93413a7ba8e9d7a4a886f83ed&show_all_topics=0&t=general&vertical_info=0%2C1%2C0%2C0%2C0%2C0%2C0%2C0%2C0%2C1

其他采集器不熟悉只会用火车这种需要抓包获取url的就懵逼了

求大佬指点或者给出个1-5页的分页url

作者: 16qf 时间: 2020-8-28 22:42

是要准备采集b乎了吗

作者: baiyangz1 时间: 2020-8-28 22:45
提示: 作者被禁止或删除内容自动屏蔽

作者: peng123 时间: 2020-8-28 22:46

没什么问题呀

作者: Alanku 时间: 2020-8-28 23:02
听说知乎反爬限制挺严，火车头真强

作者: z6045670 时间: 2020-8-28 23:35

baiyangz1 发表于 2020-8-28 22:45
http://board.locoy.com/?post=369

额就是根据这个教程没整成功

作者: z6045670 时间: 2020-8-28 23:40

peng123 发表于 2020-8-28 22:46
没什么问题呀

我一开始抓出来的也是这个url
https://www.zhihu.com/api/v4/search_v3?t=general&q=mjj&correction=1&offset=20&limit=20&lc_idx=22&show_all_topics=0&search_hash_id=557fc484503ab169f4ca6f694c5540f7&vertical_info=0%2C0%2C0%2C0%2C0%2C0%2C0%2C0%2C0%2C1

但用浏览器打开的内容是这样的啊

每个分页的url内容都是这样的

{
"error": {
"code": 10002,
"message": "10002:\u8bf7\u6c42\u53c2\u6570\u5f02\u5e38\uff0c\u8bf7\u5347\u7ea7\u5ba2\u6237\u7aef\u540e\u91cd\u8bd5"
}
}

我是需要操作什么吗？求科普感觉离真相越来越近了（我用火车头采集）

作者: 518 时间: 2020-8-28 23:55
offset=0 试试

作者: z6045670 时间: 2020-8-29 00:01

518 发表于 2020-8-28 23:55
offset=0 试试

https://www.zhihu.com/api/v4/search_v3?t=general&q=mjj&correction=1&offset=0&limit=20&lc_idx=22&show_all_topics=0&search_hash_id=557fc484503ab169f4ca6f694c5540f7&vertical_info=0%2C0%2C0%2C0%2C0%2C0%2C0%2C0%2C0%2C1

一样啊

作者: 518 时间: 2020-8-29 00:03

z6045670 发表于 2020-8-29 00:01
https://www.zhihu.com/api/v4/search_v3?t=general&q=mjj&correction=1&offset=0&limit=20&lc_idx=22&sh ...

https://api.zhihu.com/search_v3?advert_count=0&correction=1&lc_idx=0&imit=20&offset=0&q=mjj&search_hash_id=3a30b7f93413a7ba8e9d7a4a886f83ed&show_all_topics=0&t=general&vertical_info=0%2C1%2C0%2C0%2C0%2C0%2C0%2C0%2C0%2C1

作者: 爱因斯坦 时间: 2020-8-29 00:09
采集知乎没啥意义
不如头条和wx

作者: z6045670 时间: 2020-8-29 10:01