FAQ

I am getting failed to refresh token, oauth2, server_error errors when trying to access the cluster with kubectl.

Download the config file from the portal again and replace the old one. See Getting Started

This happens too often, and I need to pull the config file over and over again.

You are probably using kubectl concurrently (from several shells in parallel), which breaks the token update mechanism. Consider using ServiceAccounts for scripts. Also it’s possible CILogon blocked your IP for abusing their service (if you were using the config via some scripts and some library tried to update the token too frequently).

I downloaded the config file from the portal and suddenly became guest.

This may happen if you chose a different institution during CILogon login than when you were added to a namespace. Even if UCSD is using Google for AD accounts, for CILogon Google and UCSD are two different institutions, which would result in two different accounts.

My pod is stuck Terminating.

This happens for a few reasons, such as:
The node running your pod went offline. The pod will finish terminating once the node is back online.
The storage attached to the pod can’t be unmounted.
Due to the high load on the node, your pod termination process could not be completed. In all these cases, you should ask a cluster admin in Matrix chat to look at your pod, or just wait for somebody to fix it.
Do not use kubectl delete --grace-period=0 --force to delete stuck pods!
It will keep resources attached to the node for an indefinite period and will require the rebooting of the node (if you happen to know which node was it).

I tried to use nvprof in my GPU pod and got an error.

There is a vulnerability in NVIDIA drivers still not fixed, and this feature is disabled by default. Enabling it requires too much effort, so for now we keep it default. Hopefully it will be fixed soon.

How do I acknowledge support from NRP / Natulius in research papers?

This work was supported in part by National Science Foundation (NSF) awards CNS-1730158, ACI-1540112, ACI-1541349, OAC-1826967, OAC-2112167, CNS-2100237, CNS-2120019.