当前位置:网站首页>Tiktok crawler, what do you do when you write a crawler to catch APP request packets?

Tiktok crawler, what do you do when you write a crawler to catch APP request packets?

2020-12-06 10:09:11 TiToData


The main content of this article is to solve the problem of encountering APP Not used Android Self contained HTTP The client requests , And right HTTP The client code is confused , Lead to general tools JustTrustMe Failure problem . And in the middle level, besides JustTrustMe All other methods will not be able to deal with this situation , The reason is that in the intermediate article 1、3、4 The method is essentially aimed at Android 7.0+ System added SSL Pinning programme , And can't deal with each other HTTP The detection scheme implemented by the client itself takes effect .( I heard that there was a speed camera APP This is the type )
So what should be done to catch this kind of APP What about the bag ? It's simple , Still in use JustTrustMe And so on. Hook The tools will be fine , It's just that we need to address the confusion of the name to the original Hook Special treatment for parts of .


I've written a sample here APP To demonstrate ( Don't catch me , I don't know anything ), This APP What you do is after you click the button , A request for Baidu home page , But the request has not been cracked SSL Pinning Under normal circumstances, it is impossible to succeed , Because I set a random certificate hash , Therefore, the request fails because the normal certificate hash is different from the hash I randomly input .
image.png
Examples APP Code
This APP I've compiled it and put it in GitHub Yes , There are two versions , One is the confusion of code , One is not confused , There will be a download address at the end of the article , Readers can download and play by themselves .
image.png
Two compiled APK
Let's talk about the configuration of the tester before we start the demonstration , The testing machine used here is Android 8.1.0 Of , already Root+Xposed, At the same time, it has been installed and activated JustTrustMe.
image.png
Tester system information
image.png
Xposed Module management interface -JustTrustMe Enabled


Let's take a look at the version of the code that hasn't been confused , Put it on and open it , And then click “ Click send request ” Button .
image.png
Examples APP Interface
No surprise, the word "successful request" will appear , If your request fails , Certificate problems will prompt “ Certificate validation failed ”.
Next, let's look at the versions of the code that have been mixed up , Ibid .
image.png
Certificate validation failed
This time, the certificate verification failed ,JustTrustMe It doesn't work properly .


We'll take these two APK in Jadx In the decompile look at .
image.png stay Jadx Decompile two examples in APK
You can see in the confused version ,okhttp3 All the class names under have become abcd This kind of name .
Then let's take a look JustTrustMe Code for .
image.png
JustTrustMe In the code Hook okhttp3 Certificate verification part of
You can see that the code in it is right okhttp3.CertificatePinner Under class check methods Hook Of , This CertificatePinner Classes and check The method is not confused APK You can see clearly that .
image.png
Decompile detection code 1image.png
Decompile detection code 2
So now JustTrustMe In the face of confusion, the reason why the version fails is clear , Because it couldn't find this okhttp3.CertificatePinner.check, So it's impossible Hook To the detection method , Nature doesn't have any effect .
therefore ... What should I do ? Here is still a variety of methods for readers to choose from :
1、 modify JustTrustMe In the code Hook And then recompile
It's simple , Find the corresponding detection method , hold JustTrustMe In the code Hook Of className and methodName Change it to a confused name , For example, in this example of confusion APP in ,okhttp3.CertificatePinner.check Turned into okhttp3.f.a.
image.png
Decompile detection code - After confusion
We modify JustTrustMe Medium Hook part , It's also changed to f and a.
image.png
JustTrustMe Yes okhttp3 Certificate detection for Hookimage.png


modify JustTrustMe Yes okhttp3 Certificate detection for Hook
After that, compile it , Just install it on the mobile phone and replace the original one .
2、 Use Frida Conduct Hook
This method is better than the method 1 To be more convenient 、 More directly , Because if you need to modify the script, you can use it immediately , No need to recompile 、 Restart the phone or APP, I'll just take it here Thin dragon dance Big guy wrote relieve SSL Pinning Script Revise it , It's also a modification Hook okhttp3.CertificatePinner.check Part of , Change it to a confused name .
image.png
Modify the thin Jiao dance ObjectionUnpinningPlus Script
3、 Magic reform JustTrustMe, Add one that can adjust each type of HTTP Client's SSL Pinning The function of detecting the class name and method name of the part
I don't have time for this , Interested students can realize it by themselves .
4、 Magic reform JustTrustMe, Yes Hook Dynamic adaptation function is added in part , Even if developers confuse the code, they can automatically find the real detection class and method
ditto , The implementation can refer to Wechat wizard framework part Auto adaptation code for , In theory, it is one of the most convenient ways to realize it , Because it's completely automatic , No manual intervention required .
5、 Modify and decompile APP The code you get will be packaged back
I don't think anyone would use such a stupid way ... use Hook It's so convenient to do it in a way that .
After selecting any method to operate , Open the confused version of APP You can make a normal request .


At this time, there may be students to ask , How to know APP Which one is used HTTP client ? How to quickly locate the location of the detection method after confusion ?
It's simple , Let's turn off the cracker first , Connect the agent and grab a package to see the confusing version APP Request .
image.png
Packet capture view request
image.png
User-Agent part
Meet in peace SSL Pinning It's the same thing , Only one will be caught here CONNECT request , Pay attention to the... On the right headers, from User-Agent You can see this APP It uses okhttp3, So when we locate the detection part of the code in the obfuscated code , You just need to contrast okhttp3 Just look for the original code ( other HTTP The same for clients ). Yes, of course , I don't rule out some APP Will be able to User-Agent Get rid of , If from User-Agent If you can't see it , Then take a look at the decompiled source code structure , See if it looks like okhttp3 And so on, this is particularly obvious HTTP The name of the client , If you have one, just kill it .


——————————————————————————————————————————

TiToData: Professional short video 、 Live data interface service platform .

For more information, please contact : TiToData

Covering mainstream platforms : Tiktok , Well quickly , The little red book ,TikTok,YouTube

版权声明
本文为[TiToData]所创,转载请带上原文链接,感谢
https://chowdera.com/2020/12/20201206100703402c.html