gRPC 'Error: 14 UNAVAILABLE: TCP Write failed' issue

Phenomenon

最近在把公司的服务逐步从物理机部署迁移到 kubernetes 的过程中遇到了一些奇怪的问题。

其他服务都没啥问题,但是 nodejs 服务调用 grpc 服务的时候偶然会出现这种 Error,仅在 k8s 中出现:

Error: 14 UNAVAILABLE: TCP Write failed

at Object.exports.createStatusError (/app/node_modules/grpc/src/common.js:91:15)

at Object.onReceiveStatus (/app/node_modules/grpc/src/......

The 坑 of install Kong and Konga in Docker

Install Kong with PostgreSQL in docker

Create docker network

$ docker network create kong-net

Start database (psql should be 9.6!!)

$ docker run -d --name kong-database \

--network=kong-net \

-p 5432:5432 \

-e "POSTGRES_USER=kong" \

-e "POSTGRES_DB=kong" \

postgres:9.6

......

Certbot for wildcard domain

$ certbot certonly --manual -d *.foo.bar -d foo.bar --server https://acme-v02.api.letsencrypt.org/directory --preferred-challenges dns

Then following the instruction add add TXT record on your domain DNS