注意：以下文档只适用于TOP接口，请谨慎使用！
注意：以下文档只适用于TOP接口，请谨慎使用！
聚石塔控制台-运维中心-插件管理，SLS日志采集，后台会在集群上安装SLS组件。
成功集成后：
1）SLS侧会创建一个k8s-log-{集群id}的project；
2）集群中运行一个alibaba-log-controller；
3）集群中运行一个logtail的DaemonSet，每个ECS节点上都会起一个logtail进程。
点击安装，然后选择一个集群进行安装部署。
已迁移到新版控制台，旧版的部署配置中配置无效。
采集配置的操作说明：
1）创建/修改，会直接生效。如果状态变为"已生效"，说明底层logtail创建完毕，可以点击"查看日志"跳转到sls控制台查看logstore中的日志。如果状态异常，则说明配置有问题，可以根据错误提示进行调整
2）禁用，对于已生效的配置，禁用后状态变为"已保存"，此时不再进行日志收集
3）启用，对于状态为"已保存"的配置，启用后重新创建底层logtail，开始日志收集
4）删除，同时删除该条配置记录和底层logtail配置，不再进行日志收集
注：
建议自定义project以及logstore，比如每个应用一个project，应用的每个环境一个logstore。
① project名称仅支持小写字母、数字、连字符（-）；
② 必须以小写字母和数字开头和结尾；
③ 名称长度为3-63个字符；
① 日志库名称仅支持小写字母、数字、连字符（-）；
② 必须以小写字母和数字开头和结尾；
③ 名称长度为3-63个字符；
指定文件夹下所有符合文件名称的文件都会被监控到(包含所有层次的目录)，文件名称可以是完整名，也支持通配符模式匹配。
Linux文件路径只支持/开头，例：/apsara/nuwa/.../app.Log，
Windows文件路径只支持盘符开头，例如：C:\Program Files\Intel\...\*.Log；
极简模式默认每行为一条日志，每条日志时间使用采集时机器系统时间。通常情况下，极简模式可以满足大部分的日志采集需求。
某Java程序日志样例如下所示，日志中由于包含错误堆栈信息，可能一条日志会被分解成多行，因此需要设置行首正则表达式。
1 2 3 4 | <span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">[2018-05-11T20:10:16,000] [INFO] [SessionTracker] [SessionTrackerImpl.java:148] Expiring sessions</span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">java.sql.SQLException: Incorrect string value: '\xF0\x9F\x8E\x8F",...' for column 'data' at row 1</span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:84)</span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">at org.springframework.jdbc.support.AbstractFallbackSQLException</span></span> |
扩展配置示例：
1 2 3 4 5 6 7 8 9 | <span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">//注意，下面的内容为json字符串格式。目前只支持配置logBeginRegex,regex,key三个字段</span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">//logBeginRegex匹配样例的Java错误堆栈格式，正则表达式为 \[\d+-\d+-\w+:\d+:\d+,\d+]\s\[\w+]\s.*</span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">//regex和key将匹配到的日志，再通过正则分析，提取成key-value的形式；</span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">//举个例子，文本中的 [2018-05-11T20:10:16,000]被regex中的\[([^]]+)]匹配，最后变成了日志中的time字段</span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">{</span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">"logBeginRegex":"\[\d+-\d+-\d+\w+\d+:\d+:\d+,\d+]\s\[\w+]\s.*",</span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">"regex":"\[([^]]+)]\s\[(\w+)]\s\[(\w+)]\s\[([^:]+):(\d+)]\s(.*)",</span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">"key":["time", "level", "method", "file", "line", "message"]</span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">}</span></span> |
注意：
① 目前只支持配置logBeginRegex,regex,key三个字段；logBeginRegex必填；
② regex和key为选填，且二者要一起使用；
③ 正则表达式要根据应用具体打的日志来设置，如何调试正则表达式；
④ 关于参数更多细节：https://help.aliyun.com/document_detail/137902.html；
1 2 3 4 5 | <span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">再举个例子，tomcat的catalina日志打印如下格式：</span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">05-Aug-2020 18:38:04.954 INFO xxx</span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">那么，logBeginRegex行首表达式可以这么配置：</span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">"\d+-\w+-\d+\s\d+:\d+:\d+\.\d+\s.*"</span></span> |
Logtail支持的JSON日志是Object类型，可以自动提取Object首层的键作为字段名称，Object首层的值作为字段值。字段值可以是Object、Array或基本类型，如String、Number等。
1 2 | <span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">{"url": "POST /PutData?Category=YunOsAccountOpLog&AccessKeyId=U0Ujpek********&Date=Fri%2C%2028%20Jun%202013%2006%3A53%3A30%20GMT&Topic=raw&Signature=pD12XYLmGxKQ%2Bmkd6x7hAgQ7b1c%3D HTTP/1.1", "ip": "10.200.98.220", "user-agent": "aliyun-sdk-java", "request": {"status": "200", "latency": "18204"}, "time": "05/May/2016:13:30:28"}</span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">{"url": "POST /PutData?Category=YunOsAccountOpLog&AccessKeyId=U0Ujpek********&Date=Fri%2C%2028%20Jun%202013%2006%3A53%3A30%20GMT&Topic=raw&Signature=pD12XYLmGxKQ%2Bmkd6x7hAgQ7b1c%3D HTTP/1.1", "ip": "10.200.98.210", "user-agent": "aliyun-sdk-java", "request": {"status": "200", "latency": "10204"}, "time": "05/May/2016:13:30:29"}</span></span> |
采集模式选择JSON模式，扩展配置则无需配置。
1 2 | <span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"># '&&'分隔了两部分内容</span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">aaa&&bbb</span></span> |
采集模式选择分隔符模式，扩展配置指定。
1 2 | <span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">//注意，下面的内容为json字符串格式；目前仅支持separator和key；key的值为list</span></span><span class="lake-preview-line" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"><span class="lake-preview-line-number lake-lm-pad-level-0" style="color: rgb(191, 191, 191); margin: 0px 8px 0px 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);"></span><span class="lake-preview-codeblock-content" style="color: rgb(89, 89, 89); margin: 0px; padding: 0px; background: none 0% 0% / auto repeat scroll padding-box border-box rgba(0, 0, 0, 0);">{"separator":"&&","key":["key1","key2"]}</span></span> |
