Python 爬虫利器三之 Xpath 语法与用法2022-04-04 18:07:27

安装

pip install lxml

初步使用

首先我们利用它来解析 HTML 代码，先来一个小例子来感受一下它的基本用法。

from lxml import etree
text = '''
<div>
    <ul>
         <li class="item-0"><a href="link1.html">first item</a></li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-inactive"><a href="link3.html">third item</a></li>
         <li class="item-1"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a>
     </ul>
 </div>
'''
html = etree.HTML(text)
result = etree.tostring(html)
print(result)

首先我们使用 lxml 的 etree 库，然后利用 etree.HTML 初始化，然后我们将其打印出来。其中，这里体现了 lxml 的一个非常实用的功能就是自动修正 html 代码，大家应该注意到了，最后一个 li 标签，其实我把尾标签删掉了，是不闭合的。不过，lxml 因为继承了 libxml2 的特性，具有自动修正 HTML 代码的功能。所以输出结果是这样的

<html><body>
<div>
    <ul>
         <li class="item-0"><a href="link1.html">first item</a></li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-inactive"><a href="link3.html">third item</a></li>
         <li class="item-1"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>
</ul>
 </div>

</body></html>

不仅补全了 li 标签，还添加了 body，html 标签。

文件读取

除了直接读取字符串，还支持从文件读取内容。比如我们新建一个文件叫做 hello.html，内容为

<div>
    <ul>
         <li class="item-0"><a href="link1.html">first item</a></li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-inactive"><a href="link3.html"><span class="bold">third item</span></a></li>
         <li class="item-1"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>
     </ul>
 </div>

利用 parse 方法来读取文件。

from lxml import etree
html = etree.parse('hello.html')
result = etree.tostring(html, pretty_print=True)
print(result)

同样可以得到相同的结果。

XPath 实例测试

依然以上一段程序为例（1）获取所有的

标签

from lxml import etree
html = etree.parse('hello.html')
print type(html)
result = html.xpath('//li')
print result
print len(result)
print type(result)
print type(result[0])

运行结果

<type 'lxml.etree._ElementTree'>
[<Element li at 0x1014e0e18>, <Element li at 0x1014e0ef0>, <Element li at 0x1014e0f38>, <Element li at 0x1014e0f80>, <Element li at 0x1014e0fc8>]
5
<type 'list'>
<type 'lxml.etree._Element'>

可见，etree.parse 的类型是 ElementTree，通过调用 xpath 以后，得到了一个列表，包含了 5 个

元素，每个元素都是 Element 类型（2）获取

标签的所有 class

result = html.xpath('//li/@class')
print result

运行结果

['item-0', 'item-1', 'item-inactive', 'item-1', 'item-0']

（3）获取

标签下 href 为 link1.html 的标签

1 2	result = html.xpath('//li/a[@href="link1.html"]') print result

运行结果

1	[<Element a at 0x10ffaae18>]

（4）获取

标签下的所有标签注意这么写是不对的
1
result = html.xpath('//li/span')
因为 / 是用来获取子元素的，而并不是

的子元素，所以，要用双斜杠

1 2	result = html.xpath('//li//span') print result

运行结果

1	[<Element span at 0x10d698e18>]

（5）获取

标签下的所有 class，不包括

1 2	result = html.xpath('//li/a//@class') print result

运行结果

['blod']

（6）获取最后一个

的的 href
1
2
result = html.xpath('//li[last()]/a/@href')
print result
运行结果
1
['link5.html']
（7）获取倒数第二个元素的内容
1
2
result = html.xpath('//li[last()-1]/a')
print result[0].text
运行结果
1
fourth item
（8）获取 class 为 bold 的标签名
1
2
result = html.xpath('//*[@class="bold"]')
print result[0].tag
运行结果
1
span
通过以上实例的练习，相信大家对 XPath 的基本用法有了基本的了解。也可以利用 text 方法来获取元素的内容。大家多加练习！

结语
XPath 是一个非常好用的解析方法，同时也作为爬虫学习的基础，在后面的 selenium 以及 scrapy 框架中都会涉及到这部分知识，希望大家可以把它的语法掌握清楚，为后面的深入研究做好铺垫。

=

[ 取li text

result = html.xpath('/html/body/div[1]/div/div[1]/ul[1]/li[3]')

print (result[0].xpath('string(.)').strip())

]

=

[取a text

result = html.xpath('/html/body/div[1]/div/div[1]/ul[1]/li[3]/a/text()')

print (result)

]

=

[取a href

result = html.xpath('/html/body/div[1]/div/div[1]/ul[1]/li[3]/a/@href')

print (result)

]

文件自动加https2022-03-28 17:28:25

PHP laravel框架实现SMTP发送邮箱2022-03-21 15:54:50

首先在.env配置里配置信息

MAIL_DRIVER=smtp//主机

MAIL_HOST=smtp.163.com//主机网址

MAIL_PORT=465//端口号

MAIL_USERNAME=用户名

MAIL_PASSWORD=授权码(不是密码)

MAIL_ENCRYPTION=ssl//安全码

在控制器里引入

use Illuminate\Support\Facades\Mail;
再写一个方法,直接走一个方法里的程序就ok
class WyController extends BaseController
{
    public function r(){
        Mail::raw("范志超超超超超",function ($mesage){
            $mesage->from("17601079705@163.com","范志超");
            $mesage->subject("一二三四");
            $mesage->to("904712081@qq.com");
        });
    }
}

or

public function send()

{

$name = '学院君';

$flag = Mail::send('emails.test',['name'=>$name],function($message){

$to = '1072155122@qq.com';

$message ->to($to)->subject('测试邮件');

});

if($flag){

echo '发送邮件成功，请查收！';

}else{

echo '发送邮件失败，请重试！';

}

html中如何使一个table或者div永远处于中间位置2022-03-21 15:51:43

tml中使一个table或者div永远处于中间位置,首先需要理解块元素的概念，如div、table都是width和height属性，然后在通过css中的一个margin属性，来实现居中，具体用法看下图：

不管浏览器的宽度是多少，这个属性就会将div居中，具体看下代码：

<html>

<head>

<style>

#round{

width:400px; //通过给divwidth和heigh值，在通过margin属性来定位

height:200px;

margin:0 auto;

}

table{ //通过给divwidth和heigh值，在通过margin属性来定位

width:300px;

height:50px;

margin:0 auto;

}

</style>

</head>

<body>

</div>

<table>

<tr>

</tr>

</table>

</body>

</html>

div中内容上下居中小结2022-03-21 15:45:51

虽然div中内容上下居中的问题已经是一个比较古老的话题，但是最近发现还是有很多前端开发者在询问如何实现。其实网络上已经有很多资料和案例了，我这里再总结一下几个比较常见的处理方式。

情形一：div限高，内容长度限一行

<style>
.v-align {
margin: 0 auto;
width: 200px;
height: 80px;
text-align: center;
line-height: 80px;
border: 1px solid #ddd;
}
</style>
<!-- html -->
<div class="v-align">我的内容只能有一行。</div>

情形二：div限高，内容不限

.v-mult {
margin: 0 auto;
width: 200px;
height: 100px;
border: 1px solid #ddd;
overflow: hidden;
}
.v-mult .empty,
.v-mult .text {
display: inline-block;
*display: inline;
*zoom: 1;
vertical-align: middle;
}
.v-mult .empty {
height: 100%;
}
<!-- html -->
<div class="v-mult">
<span class="empty"></span>
<span class="text">我的内容不限，多高都行
换行照常</span>
</div>

情形三：div高度不定，内容高度一定

.v-auto {
position: relative;
margin: 0 auto;
width: 200px;
border: 1px solid #ddd;
}
.v-auto .text {
position: absolute;
top: 50%;
margin-top: -50px;
height: 100px;
border: 1px dashed #ddd;
}
<!-- html -->
<div class="v-auto">
<div class="text">
我的高度是固定的，只有100px高，但是我的父及高度不定，我怎么垂直居中呢？
</div>





</div>

情形四：div高度不定，内容高度不定

.v-auto-out { position: relative; margin: 0 auto; width: 200px; border: 1px solid #ddd; } .v-auto-out .auto-in { position: absolute; top: 50%; border: 1px dashed #ddd; /* 这里有兼容性问题 */ -webkit-transform: translateY(-50%); -ms-transform: translateY(-50%); -o-transform: translateY(-50%); transform: translateY(-50%); }  <div class="v-auto-out"> <div class="auto-in">我的高度不定，我的父及高度也不定，这下要上下居中，该如何是好？我们一起来瞧瞧吧。</div> </div>

DigitalOcean忘记root密码怎么办？2022-03-17 13:30:00

DigitalOcean忘记root密码怎么办？在这个时候我们可以选择重置.

方法1

Access->

Reset root password

Clicking Reset Root Password below will immediately shut down your Droplet and set a new root password.

The new root password will be emailed to you within a few minutes. If the email doesn't arrive or the new password doesn't work, try using the recovery environment.

Do you wish to proceed?

方法2

Access->

Recovery Console

Use the Recovery Console if you need to use the recovery ISO or you can't connect to your Droplet with the Droplet Console. To use the recovery console, you must enable password authentication. If necessary, you can reset your root password below.

Connection could not be established with host smtp.gmail.com :stream_socket_client(): unable to conn2022-03-16 10:42:16

====

[2022-03-16 02:32:13] local.ERROR: Connection could not be established with host smtp.gmail.com :stream_socket_client(): unable to connect to ssl://smtp.gmail.com:465 (Connection timed out) {"exception":"[object] (Swift_TransportException(code: 0): Connection could not be established with host smtp.gmail.com :stream_socket_client(): unable to connect to ssl://smtp.gmail.com:465 (Connection timed out) at /www/wwwroot/card.inkstarbox.com/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/StreamBuffer.php:261)

[stacktrace]

#0 [internal function]: Swift_Transport_StreamBuffer->{closure}(2, 'stream_socket_c...', '/www/wwwroot/ca...', 264, Array)

#1 /www/wwwroot/card.inkstarbox.com/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/StreamBuffer.php(264): stream_socket_client('ssl://smtp.gmai...', 110, 'Connection time...', 30, 4, Resource id #679)

#2 /www/wwwroot/card.inkstarbox.com/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/StreamBuffer.php(58): Swift_Transport_StreamBuffer->establishSocketConnection()

#3 /www/wwwroot/card.inkstarbox.com/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/AbstractSmtpTransport.php(143): Swift_Transport_StreamBuffer->initialize(Array)

#4 /www/wwwroot/card.inkstarbox.com/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Mailer.php(65): Swift_Transport_AbstractSmtpTransport->start()

#5 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Mail/Mailer.php(521): Swift_Mailer->send(Object(Swift_Message), Array)

#6 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Mail/Mailer.php(288): Illuminate\\Mail\\Mailer->sendSwiftMessage(Object(Swift_Message))

#7 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Mail/MailManager.php(484): Illuminate\\Mail\\Mailer->send('emails.test', Array, Object(Closure))

#8 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Support/Facades/Facade.php(261): Illuminate\\Mail\\MailManager->__call('send', Array)

#9 /www/wwwroot/card.inkstarbox.com/app/Http/Controllers/DoController.php(159): Illuminate\\Support\\Facades\\Facade::__callStatic('send', Array)

#10 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Controller.php(54): App\\Http\\Controllers\\DoController->personal_info(Object(Illuminate\\Http\\Request))

#11 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/ControllerDispatcher.php(45): Illuminate\\Routing\\Controller->callAction('personal_info', Array)

#12 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Route.php(254): Illuminate\\Routing\\ControllerDispatcher->dispatch(Object(Illuminate\\Routing\\Route), Object(App\\Http\\Controllers\\DoController), 'personal_info')

#13 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Route.php(197): Illuminate\\Routing\\Route->runController()

#14 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Router.php(693): Illuminate\\Routing\\Route->run()

#15 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\\Routing\\Router->Illuminate\\Routing\\{closure}(Object(Illuminate\\Http\\Request))

#16 /www/wwwroot/card.inkstarbox.com/app/Http/Middleware/SetLangMiddleware.php(30): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#17 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): App\\Http\\Middleware\\SetLangMiddleware->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#18 /www/wwwroot/card.inkstarbox.com/vendor/laravel/jetstream/src/Http/Middleware/ShareInertiaData.php(61): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#19 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Laravel\\Jetstream\\Http\\Middleware\\ShareInertiaData->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#20 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Middleware/SubstituteBindings.php(50): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#21 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Routing\\Middleware\\SubstituteBindings->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#22 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/VerifyCsrfToken.php(78): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#23 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\VerifyCsrfToken->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#24 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/View/Middleware/ShareErrorsFromSession.php(49): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#25 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\View\\Middleware\\ShareErrorsFromSession->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#26 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(121): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#27 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Session/Middleware/StartSession.php(63): Illuminate\\Session\\Middleware\\StartSession->handleStatefulRequest(Object(Illuminate\\Http\\Request), Object(Illuminate\\Session\\Store), Object(Closure))

#28 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Session\\Middleware\\StartSession->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#29 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/AddQueuedCookiesToResponse.php(37): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#30 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Cookie\\Middleware\\AddQueuedCookiesToResponse->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#31 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Cookie/Middleware/EncryptCookies.php(67): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#32 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Cookie\\Middleware\\EncryptCookies->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#33 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#34 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Router.php(695): Illuminate\\Pipeline\\Pipeline->then(Object(Closure))

#35 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Router.php(670): Illuminate\\Routing\\Router->runRouteWithinStack(Object(Illuminate\\Routing\\Route), Object(Illuminate\\Http\\Request))

#36 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Router.php(636): Illuminate\\Routing\\Router->runRoute(Object(Illuminate\\Http\\Request), Object(Illuminate\\Routing\\Route))

#37 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Routing/Router.php(625): Illuminate\\Routing\\Router->dispatchToRoute(Object(Illuminate\\Http\\Request))

#38 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(166): Illuminate\\Routing\\Router->dispatch(Object(Illuminate\\Http\\Request))

#39 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(128): Illuminate\\Foundation\\Http\\Kernel->Illuminate\\Foundation\\Http\\{closure}(Object(Illuminate\\Http\\Request))

#40 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#41 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#42 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/TransformsRequest.php(21): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#43 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\TransformsRequest->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#44 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/ValidatePostSize.php(27): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#45 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\ValidatePostSize->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#46 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Foundation/Http/Middleware/PreventRequestsDuringMaintenance.php(86): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#47 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(167): Illuminate\\Foundation\\Http\\Middleware\\PreventRequestsDuringMaintenance->handle(Object(Illuminate\\Http\\Request), Object(Closure))

#48 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Pipeline/Pipeline.php(103): Illuminate\\Pipeline\\Pipeline->Illuminate\\Pipeline\\{closure}(Object(Illuminate\\Http\\Request))

#49 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(141): Illuminate\\Pipeline\\Pipeline->then(Object(Closure))

#50 /www/wwwroot/card.inkstarbox.com/vendor/laravel/framework/src/Illuminate/Foundation/Http/Kernel.php(110): Illuminate\\Foundation\\Http\\Kernel->sendRequestThroughRouter(Object(Illuminate\\Http\\Request))

#51 /www/wwwroot/card.inkstarbox.com/public/index.php(52): Illuminate\\Foundation\\Http\\Kernel->handle(Object(Illuminate\\Http\\Request))

#52 {main}

"}

[2022-03-16 02:32:56] local.ERROR: Connection could not be established with host smtp.gmail.com :stream_socket_client(): unable to connect to ssl://smtp.gmail.com:465 (Connection timed out) {"exception":"[object] (Swift_TransportException(code: 0): Connection could not be established with host smtp.gmail.com :stream_socket_client(): unable to connect to ssl://smtp.gmail.com:465 (Connection timed out) at /www/wwwroot/card.inkstarbox.com/vendor/swiftmailer/swiftmailer/lib/classes/Swift/Transport/StreamBuffer.php:261)