为什么 Goroutines 中的 IO 操作较慢?
来源:stackoverflow
时间:2024-04-21 17:33:34 453浏览 收藏
知识点掌握了,还需要不断练习才能熟练运用。下面golang学习网给大家带来一个Golang开发实战,手把手教大家学习《为什么 Goroutines 中的 IO 操作较慢?》,在实现功能的过程中也带大家重新温习相关知识点,温故而知新,回头看看说不定又有不一样的感悟!
从 goroutine 下载图像后,我正在处理 io。
测试过程中出现问题。
在goroutine中下载图像后,我发现io操作非常慢的情况。
相反,它会在 goroutine 中下载图像 groutine 领域之外的 io 操作速度更快。
我可以知道为什么吗?
下面是测试源码
type imageresult struct { targetimagepath string success bool } type imagedownloadresult struct { targetimagepath string response *http.response success bool } func main() { downloadurls := []string{ "https://myhost.com/item/image/path/name_01.png", "https://myhost.com/item/image/path/name_02.png", "https://myhost.com/item/image/path/name_03.png", "https://myhost.com/item/image/path/name_05.png", "https://myhost.com/item/image/path/name_07.png", "https://myhost.com/item/image/path/name_08.png", "https://myhost.com/item/image/path/name_09.png", "https://myhost.com/item/image/path/name_10.png", "https://myhost.com/item/image/path/name_11.png", "https://myhost.com/item/image/path/name_12.png", "https://myhost.com/item/image/path/name_13.png", "https://myhost.com/item/image/path/name_14.png", "https://myhost.com/item/image/path/name_16.png", } startasyncio := time.now() resultchannel := make(chan imageresult) for _, downloadurl := range downloadurls { go httpfiledownloadandiowithasync(downloadurl, resultchannel) } for i := 0; i < len(downloadurls); i++ { <-resultchannel } fmt.println("total time async io: ", time.now().sub(startasyncio)) fmt.println("=======================vs========================") startsyncio := time.now() resultchannel2 := make(chan imagedownloadresult) for _, downloadurl := range downloadurls { go httpfiledownloadwithasync(downloadurl, resultchannel2) } for i := 0; i < len(downloadurls); i++ { result := <-resultchannel2 getbytesfromdownloaddata(result.response, result.targetimagepath) } fmt.println("total time sync io: ", time.now().sub(startsyncio)) }
func httpfiledownloadandiowithasync(downloadurl string, imageresult chan imageresult) { result := imageresult{ targetimagepath: downloadurl, } client := http.client{ checkredirect: func(r *http.request, via []*http.request) error { r.url.opaque = r.url.path return nil }, } // put content on file downstart := time.now() resp, _ := client.get(downloadurl) downend := time.now() fmt.println(downloadurl, "file download time : ", downend.sub(downstart)) defer resp.body.close() // file read iostart := time.now() var buf bytes.buffer io.copy(&buf, resp.body) ioend := time.now() fmt.println(downloadurl, "io time async : ", ioend.sub(iostart)) result.success = true imageresult <- result } func httpfiledownloadwithasync(downloadurl string, imageresult chan imagedownloadresult) { result := imagedownloadresult{ targetimagepath: downloadurl, } client := http.client{ checkredirect: func(r *http.request, via []*http.request) error { r.url.opaque = r.url.path return nil }, } // put content on file downstart := time.now() resp, _ := client.get(downloadurl) downend := time.now() fmt.println(downloadurl, "file download time : ", downend.sub(downstart)) result.success = true result.response = resp imageresult <- result } func getbytesfromdownloaddata(resp *http.response, downloadurl string) []byte { defer func() { if err2 := resp.body.close(); err2 != nil { fmt.println("fail download by image download close error:", downloadurl) } }() // file read starttime := time.now() var buf bytes.buffer _, err := io.copy(&buf, resp.body) if err != nil { fmt.println("fail download by read response body:", downloadurl) return nil } fmt.println(downloadurl, "io time sync:", time.now().sub(starttime)) return buf.bytes() }
下面是日志。
https://myhost.com/item/image/path/name_13.png File Download Time : 197.058394ms https://myhost.com/item/image/path/name_12.png File Download Time : 399.633804ms https://myhost.com/item/image/path/name_08.png File Download Time : 587.309339ms https://myhost.com/item/image/path/name_08.png IO Time Async : 314.482233ms https://myhost.com/item/image/path/name_03.png File Download Time : 901.985524ms https://myhost.com/item/image/path/name_05.png File Download Time : 1.132634351s https://myhost.com/item/image/path/name_02.png File Download Time : 1.132661015s https://myhost.com/item/image/path/name_14.png File Download Time : 1.132605289s https://myhost.com/item/image/path/name_09.png File Download Time : 1.132608987s https://myhost.com/item/image/path/name_16.png File Download Time : 1.133075291s https://myhost.com/item/image/path/name_01.png File Download Time : 1.132837045s https://myhost.com/item/image/path/name_11.png File Download Time : 1.133100234s https://myhost.com/item/image/path/name_10.png File Download Time : 1.132982295s https://myhost.com/item/image/path/name_07.png File Download Time : 1.133150493s https://myhost.com/item/image/path/name_12.png IO Time Async : 1.240533838s https://myhost.com/item/image/path/name_09.png IO Time Async : 849.335303ms https://myhost.com/item/image/path/name_03.png IO Time Async : 1.080254194s https://myhost.com/item/image/path/name_02.png IO Time Async : 849.395964ms https://myhost.com/item/image/path/name_13.png IO Time Async : 1.784857595s https://myhost.com/item/image/path/name_14.png IO Time Async : 849.642554ms https://myhost.com/item/image/path/name_16.png IO Time Async : 849.494898ms https://myhost.com/item/image/path/name_01.png IO Time Async : 850.297187ms https://myhost.com/item/image/path/name_10.png IO Time Async : 864.482359ms https://myhost.com/item/image/path/name_11.png IO Time Async : 864.524354ms https://myhost.com/item/image/path/name_07.png IO Time Async : 874.676604ms https://myhost.com/item/image/path/name_05.png IO Time Async : 875.22765ms Total Time Async IO: 2.008162313s =======================VS======================== https://myhost.com/item/image/path/name_09.png File Download Time : 72.476375ms https://myhost.com/item/image/path/name_05.png File Download Time : 73.351299ms https://myhost.com/item/image/path/name_07.png File Download Time : 92.839309ms https://myhost.com/item/image/path/name_10.png File Download Time : 105.41514ms https://myhost.com/item/image/path/name_08.png File Download Time : 136.861107ms https://myhost.com/item/image/path/name_01.png File Download Time : 137.531384ms https://myhost.com/item/image/path/name_16.png File Download Time : 204.833342ms https://myhost.com/item/image/path/name_11.png File Download Time : 225.73164ms https://myhost.com/item/image/path/name_03.png File Download Time : 238.569755ms https://myhost.com/item/image/path/name_09.png IO Time Sync: 251.986344ms https://myhost.com/item/image/path/name_14.png File Download Time : 473.071003ms https://myhost.com/item/image/path/name_02.png File Download Time : 523.402477ms https://myhost.com/item/image/path/name_13.png File Download Time : 523.389256ms https://myhost.com/item/image/path/name_12.png File Download Time : 523.412647ms https://myhost.com/item/image/path/name_05.png IO Time Sync: 549.364233ms https://myhost.com/item/image/path/name_07.png IO Time Sync: 890.004µs https://myhost.com/item/image/path/name_10.png IO Time Sync: 545.761µs https://myhost.com/item/image/path/name_08.png IO Time Sync: 229.321µs https://myhost.com/item/image/path/name_01.png IO Time Sync: 601.996µs https://myhost.com/item/image/path/name_16.png IO Time Sync: 12.912227ms https://myhost.com/item/image/path/name_11.png IO Time Sync: 148.432703ms https://myhost.com/item/image/path/name_03.png IO Time Sync: 336.862µs https://myhost.com/item/image/path/name_14.png IO Time Sync: 239.328µs https://myhost.com/item/image/path/name_02.png IO Time Sync: 483.976µs https://myhost.com/item/image/path/name_13.png IO Time Sync: 215.655µs https://myhost.com/item/image/path/name_12.png IO Time Sync: 265.376µs Total Time Sync IO: 1.039298797s
正确答案
Go 代码总是在 goroutine 中执行。
Goroutine 是平等的,不区分(除非运行 main()
函数的 main()
goroutine 结束时,整个应用程序终止)。
Goroutines 被调度/复用到操作系统线程上,线程在物理或虚拟 CPU 核心上运行。因此,一个 Goroutine 是否比另一个 Goroutine 运行得更慢/更快取决于执行其指令的 CPU 核心的加载方式(系统方面)。最终执行一个 Goroutine 指令的 CPU 核心可能会比另一个 Goroutine 指令被更多地利用,从而导致感知到的 Goroutine 性能更差,但这并不是因为 Go 的运行时和 Goroutine 调度。
请注意,可以使用 runtime.LockOSThread()
将 Goroutine 锁定到操作系统线程,这意味着 Goroutine 将“拥有”该线程(不会将其他 Goroutine 调度到该线程上),但您不会在应用程序中使用它。
因此,您从其他 goroutines 遇到的下载速度较慢的情况与 Go 无关,它可能与您的操作系统和 CPU 负载或您调用的外部服务(HTTP 服务器)有关。
另请注意,再次运行相同的代码可能会花费更少的时间,初始化的代码可能会被重用,并且连接到同一主机也可能会明显更快:DNS 查找会被缓存,甚至 TCP 连接也可能会被缓存/池化。您调用的服务器也可能缓存某些数据,因此获取相同的 URL 也可能会明显更快(在后续调用中从同一服务器获取不同的资源也可能会更快,某些检查(例如身份验证/授权)可能会被缓存)。
查看相关:Order of the code and performance
以上就是本文的全部内容了,是否有顺利帮助你解决问题?若是能给你带来学习上的帮助,请大家多多支持golang学习网!更多关于Golang的相关知识,也可关注golang学习网公众号。
-
502 收藏
-
502 收藏
-
501 收藏
-
501 收藏
-
501 收藏
-
139 收藏
-
204 收藏
-
325 收藏
-
477 收藏
-
486 收藏
-
439 收藏
-
357 收藏
-
352 收藏
-
101 收藏
-
440 收藏
-
212 收藏
-
143 收藏
-
- 前端进阶之JavaScript设计模式
- 设计模式是开发人员在软件开发过程中面临一般问题时的解决方案,代表了最佳的实践。本课程的主打内容包括JS常见设计模式以及具体应用场景,打造一站式知识长龙服务,适合有JS基础的同学学习。
- 立即学习 542次学习
-
- GO语言核心编程课程
- 本课程采用真实案例,全面具体可落地,从理论到实践,一步一步将GO核心编程技术、编程思想、底层实现融会贯通,使学习者贴近时代脉搏,做IT互联网时代的弄潮儿。
- 立即学习 508次学习
-
- 简单聊聊mysql8与网络通信
- 如有问题加微信:Le-studyg;在课程中,我们将首先介绍MySQL8的新特性,包括性能优化、安全增强、新数据类型等,帮助学生快速熟悉MySQL8的最新功能。接着,我们将深入解析MySQL的网络通信机制,包括协议、连接管理、数据传输等,让
- 立即学习 497次学习
-
- JavaScript正则表达式基础与实战
- 在任何一门编程语言中,正则表达式,都是一项重要的知识,它提供了高效的字符串匹配与捕获机制,可以极大的简化程序设计。
- 立即学习 487次学习
-
- 从零制作响应式网站—Grid布局
- 本系列教程将展示从零制作一个假想的网络科技公司官网,分为导航,轮播,关于我们,成功案例,服务流程,团队介绍,数据部分,公司动态,底部信息等内容区块。网站整体采用CSSGrid布局,支持响应式,有流畅过渡和展现动画。
- 立即学习 484次学习