Help:Using OpenStack APIs - Wikitech
Jump to content
From Wikitech
Cloud VPS
Cloud Services overview
Cloud VPS user docs
Horizon web interface
help
Get started
About Cloud VPS projects
Cloud VPS user roles and rights
List of Cloud VPS projects
Instances and access
Set up a Cloud VPS instance
Access Cloud VPS instances
Add disk space to instances (detachable volumes)
Server groups
Sudo policies
Unmanaged instances
Networking
Security groups and firewall settings
Floating IP addresses
Web proxies for exposing Cloud VPS services to the internet
Sending outbound email
Cloud VPS IP space
Puppet
Puppet on Cloud VPS
Project puppetserver
Managed storage
Database instances
(Trove)
Object storage
(S3 / Swift)
Get help
Help and communication
Recommendations for moving to production
Programmatic usage
OpenStack APIs
OpenTofu
Documentation for admins
Cloud VPS infrastructure
Administrator documentation
Administrator runbooks
edit
Most Cloud VPS actions can most easily be accomplished via the
Horizon web interface
. If you prefer to use a commandline or are automating operations, you may want to access the OpenStack services directly.
The OpenStack APIs are extensively documented on the
OpenStack doc site
and elsewhere, and we will not duplicate that information here. The two cloud-vps-specific thing you absolutely need to know are:
Only authentication via OpenStack Application Credentials is supported; attempting to authenticate directly with a username and password will fail.
The auth URL for Wikimedia Cloud VPS is
Application Credentials
Never store your personal
developer account
password
in a Toolforge tool, Cloud VPS instance, or anywhere else where they might be viewed by another user. These credentials are used throughout the Wikimedia tech ecosystem, not just for Cloud VPS access. Application Credentials are safer to use as they have limited access and scope.
Even though application credentials are mostly bound to per project, they can still access some global state about your OpenStack account. If you need to store an application credential inside a shared environment, it is strongly recommended to create the credentials with a separate developer account, or at least add restrictions to limit which APIs the credentials can access.
Direct password authentication against OpenStack is prevented by the Cloud VPS auth setup. In order to access these serves outside of Horizon you will need an application credential. Application credentials provide limited-scope access to OpenStack services. A single set of credentials is limited to a particular project, and can also be restricted to a limited time period or a subset of apis or commands.
Application credentials are created in Horizon via the Identity->Application Credentials panel. You can create as many sets of credentials as you like -- it's always safer to create a new set of credentials for each use case rather than reusing one set for multiple purposes. Whenever possible, restrict your credentials to just the commands that you plan to run, and limit them to the time period in which you plan to use them.
Keep in mind that once a credential is created, you can neither edit them nor view the secret passphrase. Credentials are cheap -- if you lose access to a set, just delete and make new ones.
Interfaces
OpenStack Commandline Interfaces
OpenStack provides a
commandline tool
that can access most OpenStack endpoints. On Debian or Ubuntu, the package is named 'python3-openstackclient'.
The OpenStack CLI uses a plugin architecture; each commandline will look like
openstack
. It is possible to include credentials in the command (e.g. --os-application-credential-secret) before the service name, but in most cases you will want to add your credentials to the shell environment or include them in a script.
Here is an example where the credentials are exported to the environment before using the CLI to list the VMs in a project.
apt-get
install
python3-openstackclient
export
OS_AUTH_URL
"https://openstack.eqiad1.wikimediacloud.org:25000/v3"
export
OS_IDENTITY_API_VERSION
export
OS_AUTH_TYPE
v3applicationcredential
export
OS_APPLICATION_CREDENTIAL_SECRET

export
OS_APPLICATION_CREDENTIAL_ID
f20108fb41324be19587c48d47bf76fe
export
OS_APPLICATION_CREDENTIAL_NAME
clidemo
export
OS_INTERFACE
public
openstack
server
list
+--------------------------------------+-------------------------+--------+------------------------------------------+----------------------------------------------+-----------------------+
ID Name Status Networks Image Flavor +--------------------------------------+-------------------------+--------+------------------------------------------+----------------------------------------------+-----------------------+
cd0838db-578b-4fc8-b99b-be6300ef6a66 util-bullseye-codfw1dev ACTIVE lan-flat cloudinstances2b=172.16.128.70 debian-11.0-bullseye (deprecated 2022-06-02) g2.cores1.ram2.disk20
b6e7ea90-7072-4536-9571-072c88c2e930 util-codfw1dev ACTIVE lan-flat cloudinstances2b=172.16.128.122 debian-11.0-bullseye (deprecated 2022-06-02) g2.cores1.ram2.disk20
+--------------------------------------+-------------------------+--------+------------------------------------------+----------------------------------------------+-----------------------+
The OS_AUTH_URL environment variable (or --os-auth-url commandline argument) will always point to the public Keystone endpoint. All other endpoints are discoverable via keystone, and the OpenStack cli tool will handle that discovery and query whichever other urls are needed.
OpenStack Python libraries
OpenStack also provides ready-made python bindings for accessing APIs in python code. Bindings for each project are installed separately; for example, the Nova (aka 'compute') bindings are in the Debian package
python3-novaclient
; the keystone (aka 'identity') bindings are in
python3-keystoneclient
There are a few unobvious things about using these bindings. First, you will always need to install the Keystone client package no matter what you're doing, because Keystone handles authentication and service discovery. Also, be aware that these packages include deprecated, service-specific commandline tools along with the python bindings; those CLIs are best ignored in favor of the much-newer integrated OpenStack client described above.
Here is some sample code of a script using keystone to authenticate, and then the nova bindings to list the VMs in a project:
andrew@buster:~/openstackclientthings$
cat
appcreddemo.py
from keystoneauth1.identity.v3 import ApplicationCredential
from keystoneauth1 import session as keystone_session
from novaclient import client as nova_client
auth = ApplicationCredential(
auth_url="https://openstack.eqiad1.wikimediacloud.org:25000/v3",
application_credential_secret='',
application_credential_id='1d083ef393ea4dafb86446da920a4c61',
user_domain_id='default',
session = keystone_session.Session(auth=auth)
client = nova_client.Client(
'2', session=session, connect_retries=5, timeout=300,
region_name='codfw1dev-r')
print(client.servers.list())
andrew@buster:~/openstackclientthings$
python3
./appcreddemo.py
[, ]
Gophercloud Go library
There isn't an official OpenStack SDK for Golang, but there's a community project called
Gophercloud
which usually works fine. There's also a library called
go-cloudvps
that extends Gophercloud to work with Cloud VPS specific features.
TODO: add an example for using Gophercloud
Direct API Access
Whenever possible you will want to use the ready-made bindings or CLI rather than accessing APIs directly. Nonetheless, for curl purists, or those working in non-python languages, here is a brief overview of the process:
Use the Keystone token URL (
) to acquire a token
Parse the response for the token and the catalog. The catalog contains a list of endpoints for the different services (e.g. nova, cinder, etc.).
Include the access token in a request to the service endpoint of your choice. Routes for each endpoint are documented in the
upstream OpenStack docs
TODO: add example of using curl to get a token from an application credential
Best Practices, Realistic Expectations
Usage
Our OpenStack control plane is first and foremost designed to support interactive, human users. Many actions take seconds or minutes to perform, and there are race conditions inherent in many sequences of operations. Please keep this in mind when writing automated tooling. If you need an openstack client to act like a frenzied robot rather than a human operator clicking buttons between sips of coffee, please reach out to WMCS staff so we can coordinate and advise. We will not hesitate to block clients that monopolize or stress our infrastructure.
Uptime
Most OpenStack actions (e.g. VM creation) are infrequent and non-urgent. For this reason, the WMCS team does not prioritize API uptime as highly as things that are more public-facing like VM uptime or network access. Because API uptime is provided on a 'best effort' basis, any automated tooling or workflows that rely on the OpenStack APIs should be prepared to handle downtime gracefully and recover without human interaction; otherwise you might spend a lot of time babysitting your services during upgrade or maintenance windows (or even just middle-of-the-night outages.)
Communication and support
Support and administration of the WMCS resources is provided by the
Wikimedia Foundation Cloud Services team
and
Wikimedia movement volunteers
. Please reach out with questions and join the conversation:
Discuss and receive general support
Chat in real time in the
IRC channel
#wikimedia-cloud
connect
or the bridged
Telegram group
Discuss via email after you have subscribed to the
cloud@
mailing list
Stay aware of critical changes and plans
Subscribe to the
cloud-announce@
mailing list
(all messages are also mirrored to the
cloud@
list)
Read the
News
wiki page
Track work tasks and
report bugs
Use a subproject of the
#Cloud-Services
Phabricator
project to track confirmed bug reports and feature requests about the Cloud Services infrastructure itself
Read stories and WMCS blog posts
Read the
Cloud Services Blog
(for the broader Wikimedia movement, see the
Wikimedia Technical Blog
Retrieved from "
Categories
Cloud VPS
How-to-guide
Help
Using OpenStack APIs
Add topic