Logstash jdbc 按时间增量更新的一些总结

知识点掌握了，还需要不断练习才能熟练运用。下面golang学习网给大家带来一个数据库开发实战，手把手教大家学习《Logstash jdbc 按时间增量更新的一些总结》，在实现功能的过程中也带大家重新温习相关知识点，温故而知新，回头看看说不定又有不一样的感悟！

不同数据库的支持

mysql

sqlserver

db2

Oracle

Postgre

sybase

总结

通过测试，基本上可以断定，只要是时间格式为

mysql> desc tbl_time;
+-----------+--------------+------+-----+---------+-------+
| Field     | Type         | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+-------+
| id        | int(11)      | NO   | PRI | NULL    |       |
| bigid     | bigint(20)   | YES  |     | NULL    |       |
| name      | varchar(255) | YES  |     | NULL    |       |
| date      | date         | YES  |     | NULL    |       |
| time      | datetime     | YES  |     | NULL    |       |
| timestamp | timestamp    | YES  |     | NULL    |       |
| shjnch    | bigint(255)  | YES  |     | NULL    |       |
+-----------+--------------+------+-----+---------+-------+
7 rows in set (0.00 sec)

表中有一条数据：

mysql> select * from tbl_time;
+----+-------+-------+------------+---------------------+---------------------+------------+
| id | bigid | name  | date       | time                | timestamp           | shjnch     |
+----+-------+-------+------------+---------------------+---------------------+------------+
|  1 |     1 | time1 | 2020-10-21 | 2020-10-21 09:37:31 | 2020-10-21 09:37:34 | 1603244266 |
+----+-------+-------+------------+---------------------+---------------------+------------+
1 row in set (0.00 sec)

为了能够查询出数据，我将增量查询的

sql

>

号，便于看到效果。
input {
    jdbc {
        jdbc_driver_class => "com.mysql.jdbc.Driver"
        jdbc_connection_string => "jdbc:mysql://127.0.0.1:3306/mytest?useSSL=false"
        jdbc_user => "root"
        jdbc_password => "123456"
        statement => "select *, FROM_UNIXTIME(shjnch, '%Y-%m-%d %h:%i:%s') as timestamp from tbl_time where FROM_UNIXTIME(shjnch, '%Y-%m-%d %h:%i:%s')  "*/1 * * * *"
        connection_retry_attempts => 5
        connection_retry_attempts_wait_time => 1
        tracking_column => "timestamp"
        tracking_column_type => "timestamp"
        columns_charset => {
            "message" => "utf-8"
        }
        use_column_value => true
        lowercase_column_names => false
        record_last_run => true
        add_field => {
            "@topic" => "fc491237449424896"
            "@tags" => []
            "@ip" => "127.0.0.1"
        }
    }
}

通过kafka可消费到数据如下：

{"@topic":"fc491237449424896","timestamp":"2020-10-21 09:37:46","@ip":"127.0.0.1","date":"2020-10-21T00:00:00.000+08:00","bigid":1,"time":"2020-10-21T09:37:31.000+08:00","@timestamp":"2020-10-21T15:36:02.526+08:00","id":1,"shjnch":1603244266,"name":"time1"}

本篇关于《Logstash jdbc 按时间增量更新的一些总结》的介绍就到此结束啦，但是学无止境，想要了解学习更多关于数据库的相关知识，请关注golang学习网公众号！