Cloud-Provider-Kind gateway support for kind cluster #175
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: shrutiyam-glitch The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
✅ Deploy Preview for agent-sandbox canceled.
|
k8s-ci-robot
added
do-not-merge/work-in-progress
|
Hi @shrutiyam-glitch. Thanks for your PR. I'm waiting for a github.com member to verify that this patch is reasonable to test. If it is, they should reply with Once the patch is verified, the new status will be reflected by the I understand the commands that are listed here. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
k8s-ci-robot
added
size/L
|
/ok-to-test |
k8s-ci-robot
added
ok-to-test
k8s-ci-robot
removed
the
needs-rebase
shrutiyam-glitch
changed the title
k8s-ci-robot
removed
the
do-not-merge/work-in-progress
dev/tools/deploy-to-kube
Outdated
| subprocess.run(cmd, cwd=repo_root, check=True, input=modified_content, text=True) | ||
|
|
||
|
|
||
| if args.gateway: |
There was a problem hiding this comment.
Starting a background host process inside deploy-to-kube is not the right way to go because it complicates lifecycle management (starting/stopping) and introduces dependencies into the deployment phase.
I suggest moving the cloud-provider-kind startup logic into gateway-kind/run-test-kind.sh or a dedicated setup script. This ensures the process is started specifically for the test session and cleaned up afterwards.
clients/python/agentic-sandbox-client/sandbox-router/sandbox_router.yaml
Outdated
Show resolved
Hide resolved
clients/python/agentic-sandbox-client/python-sandbox-template.yaml
Outdated
Show resolved
Hide resolved
dev/tools/deploy-to-kube
Outdated
| go_bin = os.path.expanduser('~/go/bin') | ||
| os.environ['PATH'] = f"{path}:{go_bin}" | ||
|
|
||
| print("Starting cloud-provider-kind in the background and enabling the Gateway API controller ...") |
There was a problem hiding this comment.
Can we take this to another dev/tool that specifically sets up the kube with the cloud provider gateway?
k8s-ci-robot
added
the
needs-rebase
k8s-ci-robot
removed
the
needs-rebase
…m-glitch/agent-sandbox into contour-gateway-support
igooch
left a comment
igooch
left a comment
There was a problem hiding this comment.
Running into some issues when running the script locally:
~/agent-sandbox/clients/python/agentic-sandbox-client/gateway-kind$ ./run-test-kind.sh
...
========= ./run-test-kind.sh - Running the Python client tester... =========
Traceback (most recent call last):
File "/usr/me/agent-sandbox/clients/python/agentic-sandbox-client/./test_client.py", line 17, in <module>
from agentic_sandbox import SandboxClient
File "/usr/me/agent-sandbox/clients/python/agentic-sandbox-client/agentic_sandbox/__init__.py", line 15, in <module>
from .sandbox_client import SandboxClient
File "/usr/me/agent-sandbox/clients/python/agentic-sandbox-client/agentic_sandbox/sandbox_client.py", line 26, in <module>
from kubernetes import client, config, watch
ModuleNotFoundError: No module named 'kubernetes'
Cleaning up python-runtime and sandbox controller...
sandboxtemplate.extensions.agents.x-k8s.io "python-sandbox-template" deleted
statefulset.apps "agent-sandbox-controller" deleted
customresourcedefinition.apiextensions.k8s.io "sandboxes.agents.x-k8s.io" deleted
Deleting kind cluster...
make: *** No rule to make target 'delete-kind'. Stop.
Cleanup completed.
clients/python/agentic-sandbox-client/gateway-kind/gateway-kind.yaml
Outdated
Show resolved
Hide resolved
| echo "Applying CRD for router template" | ||
| sed -i "s|IMAGE_PLACEHOLDER|${SANDBOX_ROUTER_IMG}|g" sandbox_router.yaml | ||
| kubectl apply -f sandbox_router.yaml | ||
| sleep 60 # wait for sandbox-router to be ready |
There was a problem hiding this comment.
You can probably use kubectl rollout here since the sandbox router is a deployment.
| # echo "Setting up cloud-provider-kind for gateway ..." | ||
| # go install sigs.k8s.io/cloud-provider-kind@latest | ||
| # sudo install ~/go/bin/cloud-provider-kind /usr/local/bin | ||
|
|
||
| # echo "Starting cloud-provider-kind and enabling the Gateway API controller ..." | ||
| # cloud-provider-kind --gateway-channel standard & | ||
| # echo "Cloud Provider started." | ||
| # sleep 60 |
There was a problem hiding this comment.
What are these comments for?
| killall cloud-provider-kind | ||
| kubectl delete --ignore-not-found -f python-sandbox-template.yaml | ||
| kubectl delete --ignore-not-found statefulset agent-sandbox-controller -n agent-sandbox-system | ||
| kubectl delete --ignore-not-found crd sandboxes.agents.x-k8s.io |
There was a problem hiding this comment.
If you're deleting the cluster there's no need to separately delete what's running on it.
| kubectl delete --ignore-not-found crd sandboxes.agents.x-k8s.io | ||
| echo "Deleting kind cluster..." | ||
| cd ../../../../ | ||
| make delete-kind || true |
There was a problem hiding this comment.
This didn't seem to work locally for me when running ./run-test-kind.sh:
:~/agent-sandbox/clients/python/agentic-sandbox-client/gateway-kind$ ./run-test-kind.sh
...
Deleting kind cluster...
make: *** No rule to make target 'delete-kind'. Stop.
Cleanup completed.
~/agent-sandbox/clients/python/agentic-sandbox-client/gateway-kind$ kind get clusters
agent-sandbox
| @@ -0,0 +1,99 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
Perhaps a question for @janetkuo or @barney-s . Does it make sense to make this a go test file instead of a bash script?
My personal preference would be to make this a gateway_kind_test.go so that we can use features available to Golang like wait.PollUntilContextTimeout; not sure how that fits in to standards for this repo.
There was a problem hiding this comment.
In this repo, we're experimenting with using Python for script, instead of bash (and remove .sh or .py so that it's easier to swap later). Using go would work too.
…d.yaml Co-authored-by: igooch <[email protected]>
5 participants
Cloud Provider Kind
Cloud provider kind gateway helps in setting up the Gateway API Gateway resource within a KinD cluster, where the underlying implementation of that Gateway is handled by a cloud provider-like component designed for KinD, cloud-provider-kind(https://github.com/kubernetes-sigs/cloud-provider-kind/blob/main/README.md). This allows you to test and develop applications that utilize the Gateway API within a local KinD environment, with the Gateway behaving as if it were provisioned by a cloud provider.
After running the script ./run-test-kind.sh -
Python sdk tests related to gateway will be added in a following PR.
Previously tried methods - Contour gateway support for kind cluster (https://projectcontour.io/docs/1.33/guides/gateway-api/). It uses Envoy DaemonSet / Loadbalancer and requires port-forwarding.