Extract response header via curl in php

$ch = curl_init();
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_VERBOSE, 1);
curl_setopt($ch, CURLOPT_HEADER, 1);
// ...

$response = curl_exec($ch);

// Then, after your curl_exec call:
$header_size = curl_getinfo($ch, CURLINFO_HEADER_SIZE);
$header = substr($response, 0, $header_size);
$body = substr($response, $header_size);

https://stackoverflow.com/questions/9183178/can-php-curl-retrieve-response-headers-and-body-in-a-single-request

Change pip source in windows

Background:

Sometimes the connection to pip origin source is extremely slow.But in you country,many optional mirrors site may speed up the pip install process.

How to:

  1. goto C:\Users\[you username]\AppData\Roaming
  2. if not exist “pip” folder , create it
  3. create pip.ini file in “pip” folder
  4. copy the follow code in pip.ini
[global]
timeout = 6000  
index-url = https://pypi.douban.com/simple
trusted-host = pypi.douban.com

Use PyPDF2 to extract the PDF annotation

Background:

I use Foxit PDF reader to read PDF books and paper.as well as take some marks and comments in the PDF file.But the PDF reader do not provide the export function,maybe i can find the entry for the function..:)

So I try to use PyPDF to export the comment I written in PDF

The Code:

#!/usr/bin/env python
# -*- coding: UTF-8 -*-
from PyPDF2 import PdfFileReader
import types
 
def text_extractor(path,output_path):
    text_file = open(output_path,'w')

    with open(path, 'rb') as f:
        pdf = PdfFileReader(f)
        # get the 4 page start from 0
        page = pdf.getPage(4)
        # write the raw text in to file
        contents = page.extractText()
        text_file.write(contents)
        text_file.close()
        # print you comment/annotation text to console
        annotations = page.get("/Annots").getObject()
        for annot in annotations:
                if annot is not None:
                        if annot.getObject().get("/Contents") is not None:
                                print(annot.getObject().get("/Contents").getObject())
                        # .get("/Highlight").getObject())


 
if __name__ == '__main__':
    path = './youPDF.pdf'
    out_path = './test.txt'
    text_extractor(path,out_path)

Finally

When you fill you PDF filename and path ,and run the script . the raw test will save in test.txt file.You comments will show in the console.But the code show above,just process page 4.If you want to export all you comment.You must add a for loop to iterate all pages and extract the text!

use Fabric & GoAccess to generate website access report

Background:

I want to know the daily website(apache2/nginx etc.) access report,like some charts include the page view(PV) , IPs.

Part 1: install fabric and goaccess

  1. $ pip install fabric
  2. $ apt-get install goaccess (installation in other platform see https://goaccess.io/download)

Part 2: write down the code

#!/usr/bin/python3.6
# -*- coding: utf-8 -*-
from fabric import Connection
import time
import os

# the host connection infomation , fill you infomation in []
c = Connection(
    host="[you host]",
    user="[username]",
    connect_kwargs={
       # key_filename": "[user private file]",
        "password": "[user_password]",
    }
)
# get the start time
current_time = time.time()
date_str = time.strftime('%Y_%m_%d',time.localtime(current_time))

# download the nginx log (access.1.log means yesterday`s access log)
c.get('/var/log/nginx/access.1.log'.format(date_str) , './access.{}.log'.format(date_str))

# run the goaccess and generate the report in html format
os.system("goaccess ./access.{}.log -o report.{}.html".format(date_str,date_str))

Finally you will see you daily report like the image below

from https://goaccess.io/features

Add google website tracking tags(Google Analytics) in WordPress5.2 manually

First: find you tracking code in google Analytics

  1. goto Admin panel
  2. click “Tracnking info” under “Properly” block
  3. click “Tracking Info” and “Tracking Code”
  4. you will see you “Global Site Tag” on the right side.
  5. copy the html script !

Second: login you webserver and edit the php file

  1. cd into wordpress folder(wordpress/wp-content/themes/twentysixteen)
  2. vim header.php
  3. paste the First step code into <head>[put here]</head>
  4. save the file

Finally: visit you wordpress again the google analytics will work!

But you must edit each theme,if you change you theme.Edit the
wordpress/wp-content/themes/[twentysixteen,twentynineteen,twentyseventeen ]/header.php

I will try a more easy way to add google Analytics !

Enable BBR on vps

How to enable?

$ echo "net.core.default_qdisc=fq" >> /etc/sysctl.conf
$ echo "net.ipv4.tcp_congestion_control=bbr" >> /etc/sysctl.conf
# enable the config
$ sysctl -p
# run the command below , if you can see "bbr" in the output , you are success!
$ sysctl net.ipv4.tcp_available_congestion_control
$ sysctl net.ipv4.tcp_congestion_control

Why we use it?

It can lower you tcp network latency and higher you througthput

And more refer:

BBR is a google technology , you can see more in
https://github.com/google/bbr

And you can see more detail in
https://github.com/google/bbr/blob/master/Presentations/bbr-2017-02-08-google-net-research-summit.pdf

When WordPress meet “Access denied for user ‘root’@’localhost'”

In my case , the key is ubuntu mysql default password. When i use “sudo mysql”, will login the mysql without any password or username.

But i use command “sudo mysql_secure_installation” to change the password , it doesn`t work!

After use below command

UPDATE mysql.user
SET authentication_string=PASSWORD(‘your password’),
plugin=’mysql_native_password’
WHERE User=’root’ AND Host=’localhost’;

The error finally solved.

CloudFlare API remark

By default , CloudFlare doesn`t support CNAME record for free plan, instead of NS record.But CNAME is more flexible ! I discover the cloudflare API website and find some APIS can create a CNAME record(but i got an error by create CNAME record)

First , Get you zones id

$ curl -X GET “https://api.cloudflare.com/client/v4/zones” \
-H “X-Auth-Email: [your cloudflare email]” \
-H “X-Auth-Key: [you cloudflare API key]” \
-H “Content-Type: application/json”

Second , Create a CNAME record , But I got an error . I will try to solve it soon.

$ curl -X POST “https://api.cloudflare.com/client/v4/zones/[you zone id get via above request]/custom_hostnames” \
-H “X-Auth-Email: [you cloudflare email]” \
-H “X-Auth-Key: [you cloudflare API key]” \
-H “Content-Type: application/json” \
–data ‘{“hostname”:”[you host like test.abc.com]”,”ssl”:{“method”:”http”,”type”:”dv”,”settings”:{“http2″:”on”,”min_tls_version”:”1.2″,”tls_1_3″:”on”,”ciphers”:[“ECDHE-RSA-AES128-GCM-SHA256″,”AES128-SHA”]}}}’