qemu RX

Fork

cleanup again

celanup

disas cleanup

disassebmer update

CTX append fix

Add TARGET_VIRT_ADDR_SPACE_BITS

Merge branch 'master' into rx-decodetable

New decodetree support

Added DisasContext to extract function

update helper

cleanup

psw cleanup

cleanup psw op

cleanup fp operation

cleanup psw op

Add a standard authorization framework

The current network services now support encryption via TLS and in some
cases support authentication via SASL. In cases where SASL is not
available, x509 client certificates can be used as a crude authorization
scheme, but using a sub-CA and controlling who you give certs to. In
general this is not very flexible though, so this series introduces a
new standard authorization framework.

It comes with four initial authorization mechanisms

- Simple - an exact username match. This is useful when there is
exactly one user that is known to connect. For example when live
migrating from one QEMU to another with TLS, libvirt would use
the simple scheme to whitelist the TLS cert of the source QEMU.

- List - an full access control list, with optional regex matching.
This is more flexible and is used to provide 100% backcompat with
the existing HMP ACL commands. The caveat is that we can't create
these via the CLI -object arg yet.

- ListFile - the same as List, but with the rules stored in JSON
format in an external file. This avoids the -object limitation
while also allowing the admin to change list entries on the file.
QEMU uses inotify to notice these changes and auto-reload the
file contents. This is likely a good default choice for most
network services, if the "simple" mechanism isn't sufficient.

- PAM - delegate the username lookup to a PAM module, which opens
the door to many options including things like SQL/LDAP lookups.
-----BEGIN PGP SIGNATURE-----

iQIcBAABCAAGBQJcdVxaAAoJEL6G67QVEE/fgK4P/jxF1aTzuc0N0jD7myNi2LE4
aUqXueawNLrZKk8ZZQtEy/71/fTNJm9fVwxN9zIWITb4jTj4c3hqUnK1iVtEXXOR
e0qZdKAsVAD2cpj9PBaihn84f+hrSF1ZnuMqtt4/kfOE9QPfLRpbqCNe+7JX5Z7z
Cw8k+ATvm6Xc5QVEkc+8JTMft9lQaOdRJvldR2CVSqeFgIZWIrQ2NIG44m2GqO6b
wqqUgmaMHd4cGJ4fgTn8G74GNzp4hpXzgTC9P881zC8qc+dKKJasTq1jSR3eWYKJ
aE4Y+4EDjH/a0sP/zf97OfDzb/mGEpw8ng00g3o0qphFmSWMHVcoghB6wUZZcZGu
VcGyoH+ztDdIcoD9HwVbkzMvYt9qrtMNrMUf3p1DRRNrrwcopSYnG6gZ9ArckgD4
2VzxLrBbfS9tsWGAvOw/+c4SIxEzGQ0mcqHQeZONxLAi+zwkzuKVu973JBkvCBus
aXpct8B2aMMxZeJ1ipWksj5PvAAEjhOquDqZrUtRiP7vg4156UiAhxW9x6hK//O1
pof4jVBPZSYgrt3+PjADwK1fmPv01KkNa1dVi/Ew7fBJjGQNfJ12JlAYdSeoGG4i
ds4OBbQH+Ko40++ZGu+n7xlk/0GQEw771c2GNDoA7ehHPg++S2yvrRoVVneyicur
RYzwaPQ1nYrGAo3fAJEd
=67pz
-----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/berrange/tags/authz-core-pull-request' into staging

Add a standard authorization framework

The current network services now support encryption via TLS and in some
cases support authentication via SASL. In cases where SASL is not
available, x509 client certificates can be used as a crude authorization
scheme, but using a sub-CA and controlling who you give certs to. In
general this is not very flexible though, so this series introduces a
new standard authorization framework.

It comes with four initial authorization mechanisms

- Simple - an exact username match. This is useful when there is
exactly one user that is known to connect. For example when live
migrating from one QEMU to another with TLS, libvirt would use
the simple scheme to whitelist the TLS cert of the source QEMU.

- List - an full access control list, with optional regex matching.
This is more flexible and is used to provide 100% backcompat with
the existing HMP ACL commands. The caveat is that we can't create
these via the CLI -object arg yet.

- ListFile - the same as List, but with the rules stored in JSON
format in an external file. This avoids the -object limitation
while also allowing the admin to change list entries on the file.
QEMU uses inotify to notice these changes and auto-reload the
file contents. This is likely a good default choice for most
network services, if the "simple" mechanism isn't sufficient.

- PAM - delegate the username lookup to a PAM module, which opens
the door to many options including things like SQL/LDAP lookups.

# gpg: Signature made Tue 26 Feb 2019 15:33:46 GMT
# gpg: using RSA key BE86EBB415104FDF
# gpg: Good signature from "Daniel P. Berrange <dan@berrange.com>" [full]
# gpg: aka "Daniel P. Berrange <berrange@redhat.com>" [full]
# Primary key fingerprint: DAF3 A6FD B26B 6291 2D0E 8E3F BE86 EBB4 1510 4FDF

* remotes/berrange/tags/authz-core-pull-request:
authz: delete existing ACL implementation
authz: add QAuthZPAM object type for authorizing using PAM
authz: add QAuthZListFile object type for a file access control list
authz: add QAuthZList object type for an access control list
authz: add QAuthZSimple object type for easy whitelist auth checks
authz: add QAuthZ object as an authorization base class
hw/usb: switch MTP to use new inotify APIs
hw/usb: fix const-ness for string params in MTP driver
hw/usb: don't set IN_ISDIR for inotify watch in MTP driver
qom: don't require user creatable objects to be registered
util: add helper APIs for dealing with inotify in portable manner

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

authz: delete existing ACL implementation

The 'qemu_acl' type was a previous non-QOM based attempt to provide an
authorization facility in QEMU. Because it is non-QOM based it cannot be
created via the command line and requires special monitor commands to
manipulate it.

The new QAuthZ subclasses provide a superset of the functionality in
qemu_acl, so the latter can now be deleted. The HMP 'acl_*' monitor
commands are converted to use the new QAuthZSimple data type instead
in order to provide temporary backwards compatibility.

Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

authz: add QAuthZPAM object type for authorizing using PAM

Add an authorization backend that talks to PAM to check whether the user
identity is allowed. This only uses the PAM account validation facility,
which is essentially just a check to see if the provided username is permitted
access. It doesn't use the authentication or session parts of PAM, since
that's dealt with by the relevant part of QEMU (eg VNC server).

Consider starting QEMU with a VNC server and telling it to use TLS with
x509 client certificates and configuring it to use an PAM to validate
the x509 distinguished name. In this example we're telling it to use PAM
for the QAuthZ impl with a service name of "qemu-vnc"

$ qemu-system-x86_64 \
-object tls-creds-x509,id=tls0,dir=/home/berrange/security/qemutls,\
endpoint=server,verify-peer=yes \
-object authz-pam,id=authz0,service=qemu-vnc \
-vnc :1,tls-creds=tls0,tls-authz=authz0

This requires an /etc/pam/qemu-vnc file to be created with the auth
rules. A very simple file based whitelist can be setup using

$ cat > /etc/pam/qemu-vnc <<EOF
account requisite pam_listfile.so item=user sense=allow file=/etc/qemu/vnc.allow
EOF

The /etc/qemu/vnc.allow file simply contains one username per line. Any
username not in the file is denied. The usernames in this example are
the x509 distinguished name from the client's x509 cert.

$ cat > /etc/qemu/vnc.allow <<EOF
CN=laptop.berrange.com,O=Berrange Home,L=London,ST=London,C=GB
EOF

More interesting would be to configure PAM to use an LDAP backend, so
that the QEMU authorization check data can be centralized instead of
requiring each compute host to have file maintained.

The main limitation with this PAM module is that the rules apply to all
QEMU instances on the host. Setting up different rules per VM, would
require creating a separate PAM service name & config file for every
guest. An alternative approach for the future might be to not pass in
the plain username to PAM, but instead combine the VM name or UUID with
the username. This requires further consideration though.

Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

authz: add QAuthZListFile object type for a file access control list

Add a QAuthZListFile object type that implements the QAuthZ interface. This
built-in implementation is a proxy around the QAuthZList object type,
initializing it from an external file, and optionally, automatically
reloading it whenever it changes.

To create an instance of this object via the QMP monitor, the syntax
used would be:

{
"execute": "object-add",
"arguments": {
"qom-type": "authz-list-file",
"id": "authz0",
"props": {
"filename": "/etc/qemu/vnc.acl",
"refresh": true
}
}
}

If "refresh" is "yes", inotify is used to monitor the file,
automatically reloading changes. If an error occurs during reloading,
all authorizations will fail until the file is next successfully
loaded.

The /etc/qemu/vnc.acl file would contain a JSON representation of a
QAuthZList object

{
"rules": [
{ "match": "fred", "policy": "allow", "format": "exact" },
{ "match": "bob", "policy": "allow", "format": "exact" },
{ "match": "danb", "policy": "deny", "format": "glob" },
{ "match": "dan*", "policy": "allow", "format": "exact" },
],
"policy": "deny"
}

This sets up an authorization rule that allows 'fred', 'bob' and anyone
whose name starts with 'dan', except for 'danb'. Everyone unmatched is
denied.

The object can be loaded on the comand line using

-object authz-list-file,id=authz0,filename=/etc/qemu/vnc.acl,refresh=yes

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

authz: add QAuthZList object type for an access control list

Add a QAuthZList object type that implements the QAuthZ interface. This
built-in implementation maintains a trivial access control list with a
sequence of match rules and a final default policy. This replicates the
functionality currently provided by the qemu_acl module.

To create an instance of this object via the QMP monitor, the syntax
used would be:

{
"execute": "object-add",
"arguments": {
"qom-type": "authz-list",
"id": "authz0",
"props": {
"rules": [
{ "match": "fred", "policy": "allow", "format": "exact" },
{ "match": "bob", "policy": "allow", "format": "exact" },
{ "match": "danb", "policy": "deny", "format": "glob" },
{ "match": "dan*", "policy": "allow", "format": "exact" },
],
"policy": "deny"
}
}
}

This sets up an authorization rule that allows 'fred', 'bob' and anyone
whose name starts with 'dan', except for 'danb'. Everyone unmatched is
denied.

It is not currently possible to create this via -object, since there is
no syntax supported to specify non-scalar properties for objects. This
is likely to be addressed by later support for using JSON with -object,
or an equivalent approach.

In any case the future "authz-listfile" object can be used from the
CLI and is likely a better choice, as it allows the ACL to be refreshed
automatically on change.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

authz: add QAuthZSimple object type for easy whitelist auth checks

In many cases a single VM will just need to whitelist a single identity
as the allowed user of network services. This is especially the case for
TLS live migration (optionally with NBD storage) where we just need to
whitelist the x509 certificate distinguished name of the source QEMU
host.

Via QMP this can be configured with:

{
"execute": "object-add",
"arguments": {
"qom-type": "authz-simple",
"id": "authz0",
"props": {
"identity": "fred"
}
}
}

Or via the command line

-object authz-simple,id=authz0,identity=fred

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

authz: add QAuthZ object as an authorization base class

The current qemu_acl module provides a simple access control list
facility inside QEMU, which is used via a set of monitor commands
acl_show, acl_policy, acl_add, acl_remove & acl_reset.

Note there is no ability to create ACLs - the network services (eg VNC
server) were expected to create ACLs that they want to check.

There is also no way to define ACLs on the command line, nor potentially
integrate with external authorization systems like polkit, pam, ldap
lookup, etc.

The QAuthZ object defines a minimal abstract QOM class that can be
subclassed for creating different authorization providers.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>

hw/usb: switch MTP to use new inotify APIs

The internal inotify APIs allow a lot of conditional statements to be
cleared out, and provide a simpler callback for handling events.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

hw/usb: fix const-ness for string params in MTP driver

Various functions accepting 'char *' string parameters were missing
'const' qualifiers.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

hw/usb: don't set IN_ISDIR for inotify watch in MTP driver

IN_ISDIR is not a bit that one can request when registering a
watch with inotify_add_watch. Rather it is a bit that is set
automatically when reading events from the kernel.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

qom: don't require user creatable objects to be registered

When an object is in turn owned by another user object, it is not
desirable to expose this in the QOM object hierarchy. It is just an
internal implementation detail, we should be free to change without
exposure to apps managing QEMU.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Tested-by: Philippe Mathieu-Daudé <philmd@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

util: add helper APIs for dealing with inotify in portable manner

The inotify userspace API for reading events is quite horrible, so it is
useful to wrap it in a more friendly API to avoid duplicating code
across many users in QEMU. Wrapping it also allows introduction of a
platform portability layer, so that we can add impls for non-Linux based
equivalents in future.

Reviewed-by: Marc-André Lureau <marcandre.lureau@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>

Pull request
-----BEGIN PGP SIGNATURE-----

iQEcBAABAgAGBQJccAIFAAoJEJykq7OBq3PImM0IAMcC92GYwSL6MgQ4NrfbVfDQ
P+qgIoxlXcYNwa12kGY6rE4tgSDab9Mc5ACpmuFdr8Xj7/JOp97AccmKZ+RzYQSj
IFlkvd8GngQR1YnFGV6PIWSt7hRhKuUZMqSIDrWro/MTdiJFEMI8/e7QLGxmaEX1
gWCNSopxJUeACSJiRyfZvBGNCs23R9ptFKBFhIXS98KPtEtF8LQV0JnQXoRUDiBL
G9C/xggdGDvct3Id4yOBCh43ErssyOrlYwjzIRWB2AFfIwHGytJcL6JBjAqy4Z7M
ClMNXaQSbfSCXrc3osF9nO6KaFduhfpUH44lL5JBHxUgH6pN2xGradRANZwysQM=
=DSBn
-----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stefanha/tags/block-pull-request' into staging

Pull request

# gpg: Signature made Fri 22 Feb 2019 14:07:01 GMT
# gpg: using RSA key 9CA4ABB381AB73C8
# gpg: Good signature from "Stefan Hajnoczi <stefanha@redhat.com>" [full]
# gpg: aka "Stefan Hajnoczi <stefanha@gmail.com>" [full]
# Primary key fingerprint: 8695 A8BF D3F9 7CDA AC35 775A 9CA4 ABB3 81AB 73C8

* remotes/stefanha/tags/block-pull-request: (27 commits)
tests/virtio-blk: add test for DISCARD command
tests/virtio-blk: add test for WRITE_ZEROES command
tests/virtio-blk: add virtio_blk_fix_dwz_hdr() function
tests/virtio-blk: change assert on data_size in virtio_blk_request()
virtio-blk: add DISCARD and WRITE_ZEROES features
virtio-blk: set config size depending on the features enabled
virtio-net: make VirtIOFeature usable for other virtio devices
virtio-blk: add "discard" and "write-zeroes" properties
virtio-blk: add host_features field in VirtIOBlock
virtio-blk: add acct_failed param to virtio_blk_handle_rw_error()
hw/ide: drop iov field from IDEDMA
hw/ide: drop iov field from IDEBufferedRequest
hw/ide: drop iov field from IDEState
tests/test-bdrv-drain: use QEMU_IOVEC_INIT_BUF
migration/block: use qemu_iovec_init_buf
qemu-img: use qemu_iovec_init_buf
block/vmdk: use qemu_iovec_init_buf
block/qed: use qemu_iovec_init_buf
block/qcow2: use qemu_iovec_init_buf
block/qcow: use qemu_iovec_init_buf
...

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

Various testing fixes:

- Travis updates (inc disable isapc cdrom test)
- Add gitlab control
- Fix docker image
- keep softloat tests short
-----BEGIN PGP SIGNATURE-----

iQEzBAABCgAdFiEEZoWumedRZ7yvyN81+9DbCVqeKkQFAlxvxigACgkQ+9DbCVqe
KkQFRAf/fv1WDebvt0MrMpRFRxRG/F8laULNFXpxkYC6Vz/eFoSlR3aimkwXlhra
ddhKAmGRARERXVzoGHMcYyhQxu3+1u6bm8lVv+OPTVD1uFY/+6Or1xxNjgIO4CeP
EZUhVKlvphXZVHdDL53sCu8aFOwZL0yHb3OqxSri7r5OsQyw75433WjAS1v2emr8
pm9N4oH7nby3tpkuFIRJLi5NvMulrTHTDmMTcpaEfNjMjc9nC5J0eCmrJe9SIi4V
VGg+rWpmerjAiQLZuM6uzQ3TR5QrDLnKP0tYDdVJaPHMIrExjDdPQxsFTbWibWfV
hCDd88m/YZMi8ERsrJ6oGewb7tcd2A==
=pTDl
-----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/stsquad/tags/pull-testing-next-220219-1' into staging

Various testing fixes:

- Travis updates (inc disable isapc cdrom test)
- Add gitlab control
- Fix docker image
- keep softloat tests short

# gpg: Signature made Fri 22 Feb 2019 09:51:36 GMT
# gpg: using RSA key 6685AE99E75167BCAFC8DF35FBD0DB095A9E2A44
# gpg: Good signature from "Alex Bennée (Master Work Key) <alex.bennee@linaro.org>" [full]
# Primary key fingerprint: 6685 AE99 E751 67BC AFC8 DF35 FBD0 DB09 5A9E 2A44

* remotes/stsquad/tags/pull-testing-next-220219-1:
tests/cdrom-test: only include isapc cdrom test when g_test_slow()
tests/softfloat: always do quick softfloat tests
Add a gitlab-ci file for Continuous Integration testing on Gitlab
tests/docker: peg netmap code to a specific version
tests/docker: squash initial update and install step for debian9
.travis.yml: Remove disable-uuid
.travis.yml: Test with disable-replication
.travis.yml: split debug builds
.travis.yml: the xcode10 image seems to be hosed

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>

vga: bugfixes and edid support for virtio-vga
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.22 (GNU/Linux)

iQIcBAABAgAGBQJcb7G5AAoJEEy22O7T6HE47MsQANkX98h9JT78GcBhO3UqE1yW
TDwi6Qj0lUEYdW08LraSIsbDC8XHMmOr35aDEdpUhZyMP1mKmnKaxm+x6YR/q01W
vExy8nMtyueHSrcJVTe5p+jS5SHXnJlJOvqL/qoh97g0zEbQXnncoBcYdzJGb5DS
AQGJ/cs+0macNeMcYqauowJONFirKDapntmdCvIHoqZzMhefiyB31U0q6GiC0aiW
70GjpSX2Mkizit4Chny1Q2DCkJchtM7qwWdVLOLbH8NMsGO3rp9PVVkMMwBGxjyP
MmT7GVf5cinXfLQGKDuDrgztHjrO2kxRH0St9TBwUVCvMY/zjtVtTc0CabeH+wDd
2joWGwRPsnYcY6MMAiFoPo0CMXhQhQjVdrqOpn8+L6rs9sdbefk7ZA5fIJoIhvjp
K5iJ8bCYIdA2R5EdKCKdAariASDAw2/ztqS3WAlpz7gAO2RMJEcqsYlUUXnl4HbB
0TFuPJpJrCnl5yv/zoKdM8cKyZza/7ULPh4uGfiB7mpcik/UfzkO0VLMuk+7xwJ/
KoNS4vHZrD7lFDHyvkgHYP7CV12ClaYhEVPWZMZ0Vz7XeVWEe/un1oyryORPSBIO
kRiqSBCZbolvJlo9Iq3gM8jIFguPD7cGzQoyr+1RaCFX6NT4kqw75n5LW+Pg6CRU
l25LOigqtUJgKBJKQD/S
=sx1C
-----END PGP SIGNATURE-----

Merge remote-tracking branch 'remotes/kraxel/tags/vga-20190222-pull-request' into staging

vga: bugfixes and edid support for virtio-vga

# gpg: Signature made Fri 22 Feb 2019 08:24:25 GMT
# gpg: using RSA key 4CB6D8EED3E87138
# gpg: Good signature from "Gerd Hoffmann (work) <kraxel@redhat.com>" [full]
# gpg: aka "Gerd Hoffmann <gerd@kraxel.org>" [full]
# gpg: aka "Gerd Hoffmann (private) <kraxel@gmail.com>" [full]
# Primary key fingerprint: A032 8CFF B93A 17A7 9901 FE7D 4CB6 D8EE D3E8 7138

* remotes/kraxel/tags/vga-20190222-pull-request:
display/virtio: add edid support.
virtio-gpu: remove useless 'waiting' field
virtio-gpu: block both 2d and 3d rendering
virtio-gpu: remove unused config_size
virtio-gpu: remove unused qdev

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>