2025年温故OpenStack中的测试(by quqi99)

大家好，我是讯享网，很高兴认识大家。

沿用tox调用virtualenv自动创建的虚拟环境(virtualenv -p python3.5 .tox/py35)

source .tox/py35/bin/activate sudo pip install --upgrade -r requirements.txt sudo pip install --upgrade -r test-requirements.txt

讯享网

使用unittest和nose运行测试。nose是对unittest的扩展，使得python的测试更加简单，nose自动发现测试代码并执行，nose提供了大量的插件，比如覆盖报表等。

讯享网python -m unittest -v unit_tests.test_neutron_utils.TestNeutronUtils.test_get_packages_ovs_newton .tox/py35/bin/python nosetests -v unit_tests/test_neutron_utils.py:TestNeutronUtils.test_get_packages_ovs_newton

注意：上面采用nosetests运行时会报错，因为我们的测试采用了python3, 所以需要在安装了python3-nose之后（sudo apt-get install python3-nose python3-mock）再采用下列三种方式之一运行：

nosetests3 -v unit_tests/test_neutron_utils.py:TestNeutronUtils.test_restart_map_ovs_odl /bak/work/charms/neutron-gateway/.tox/py35/bin/python /usr/local/bin/nosetests -v unit_tests/test_neutron_utils.py:TestNeutronUtils.test_get_packages_ovs_newton python -m nose unit_tests/test_neutron_utils.py:TestNeutronUtils.test_restart_map_ovs_odl

但实际上仍然找不找nose模块，那是因为nose与virtualenv结合地不大好，在这个网页找着了答案(https://stackoverflow.com/questions//problems-using-nose-in-a-virtualenv) - You need to have a copy of nose installed in the virtual environment. In order to force installation of nose into the virtualenv, even though it is already installed in the global site-packages, run pip install with the -I flag: pip install nose -I

上面使用unittest与nose运行测试的方式只是将结果输出到stdout，不便于分析。所以可以使用python-subunit模块来运行测试，并将测试结果通过subunit协议输出到文件中便于日后分析。因为subunit是基于二进制的不便于人眼看，所以可使用subunit2pyunit工具将其人类可读化。

讯享网python -m subunit.run discover |subunit2pyunit python -m subunit.run discover -t ./ ./unit_tests |subunit2pyunit python -m subunit.run unit_tests.test_neutron_utils.TestNeutronUtils. |subunit2pyunit

在大型应用中分析测试结果很重要，testrepository可以调用subunit来用python-subunit模块来运行测试，并将测试结果通过subunit协议输出到文件中，然后testrepository在些基础上有更多的分析，如分析哪些用例运行的时间最长，如显示失败的用例，如仅运行上次运行失败的用例。

testr init testr run testr run --parallel $ cat .testr.conf [DEFAULT] test_command=OS_STDOUT_CAPTURE=${OS_STDOUT_CAPTURE:-1} \ OS_STDERR_CAPTURE=${OS_STDERR_CAPTURE:-1} \ OS_TEST_TIMEOUT=${OS_TEST_TIMEOUT:-60} \ ${PYTHON:-python} -m subunit.run discover -t ./ ./unit_tests $LISTOPT $IDOPTION test_id_option=--load-list $IDFILE test_list_option=--list

tox用于创建虚拟python环境，也可以集成上面的testrepository(commands = ostestr {posargs})

讯享网$ cat tox.ini [tox] envlist = pep8,py27,py35 skipsdist = True [testenv] setenv = VIRTUAL_ENV={envdir} PYTHONHASHSEED=0 CHARM_DIR={envdir} AMULET_SETUP_TIMEOUT=5400 install_command = pip install --allow-unverified python-apt {opts} {packages} commands = ostestr {posargs} whitelist_externals = juju passenv = HOME TERM AMULET_* CS_API_* [testenv:py27] basepython = python2.7 deps = -r{toxinidir}/requirements.txt -r{toxinidir}/test-requirements.txt commands = /bin/true [testenv:py35] basepython = python3.5 deps = -r{toxinidir}/requirements.txt -r{toxinidir}/test-requirements.txt [testenv:pep8] basepython = python2.7 deps = -r{toxinidir}/requirements.txt -r{toxinidir}/test-requirements.txt commands = flake8 {posargs} hooks unit_tests tests actions lib charm-proof [flake8] ignore = E402,E226 exclude = */helpers

pydev使用virtualenv中的py35
在eclipse的"Preferences -> Pydev -> Interpreters -> Python Interpreters"菜单中定义python35=/bak/work/charms/neutron-gateway/.tox/py35/bin/python,然后在工程上点右键从"Properties -> Pydev - Interpreter/Grammar"定义使用python35。注意，需要将/bak/work/charms/neutron-gateway/.tox/py35/lib/python3.5/site-packages也选到环境变量中，否则后面会报ImportError: No module named 'mock。
为一个测试类定义"Python unitest"类型的"Debug Configurations", 也在其Interpreter选项卡中定义使用python35 (结果：eclipse似乎有bug，此处选择了python35后无法保存)
所以无法成功，似乎是pydev与python3协作不大好。最后还是pudb好使(sudo pip install pudb, import pudb; pdb.set_trace())
py27下的测试运行方法(如openstack), charm似乎只能用py36 (tox -r -epy36 && tox -e py36).

cat tox.ini tox -r -epy27 tox -e py27,pep8 tox -e py27 neutron.tests.unit.agent.linux.test_keepalived tox -e py27 neutron.tests.unit.agent.linux.test_keepalived.KeepalivedInstanceTestCase.test_remove_addresses_by_interface mkdir /opt/stack && sudo chown -R hua /opt/stack tox -e functional neutron.tests.functional.agent.l3.test_ha_router.L3HATestCase.test_keepalived_configuration tox -e dsvm-fullstack source .tox/functional/bin/activate .tox/functional/bin/pip install -r requirements.txt .tox/functional/bin/pip install -r test-requirements.txt .tox/functional/bin/pip install -r neutron/tests/functional/requirements.txt .tox/functional/bin/pip freeze |grep neutron .tox/functional/bin/pip install 'neutron-lib==1.13.0' OS_SUDO_TESTING=True .tox/functional/bin/python -m unittest -v neutron.tests.functional.agent.l3.test_ha_router.L3HATestCase.test_keepalived_configuration

有时如mitaka已经eol了, 例如它的origain-stable-mitaka这个分支都没有了, 这会导致运行’tox -r -epy27 '不成功, 那么可以手工执行:

讯享网virtualenv venv_mitaka . venv_mitaka/bin/activate # pg_config executable not found # Error: could not determine PostgreSQL version from '10.5' sudo apt install python-setuptools python-dev libpython-dev libssl-dev python-pip libmysqlclient-dev libxml2-dev libxslt-dev libxslt1-dev libpq-dev git git-review libffi-dev gettext graphviz libjpeg-dev zlib1g-dev build-essential python-nose python-mock libssl1.0 sudo apt install python3.6 python3.6-dev python3-pip python3-dev python3-nose python3-mock #sudo pip install --upgrade setuptools #sudo pip3 install --upgrade setuptools #sudo pip install --upgrade --force-reinstall pip virtualenv # Failed to install Cryptography - sudo apt install libssl1.0 # No module named dulwich - ./venv/bin/pip install dulwich # unittest has no attribute 'virt' - # ./venv_ocata/bin/pip install -c https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt?h=stable/ocata . ./venv_mitaka/bin/pip install -c https://git.openstack.org/cgit/openstack/requirements/plain/upper-constraints.txt?h=mitaka-eol . ./venv_mitaka/bin/pip install -r requirements.txt ./venv_mitaka/bin/pip install -r test-requirements.txt find . -name "*.pyc" -exec rm -rf {} \; git clone https://github.com/openstack/oslo.cache.git cd oslo.cache && git checkout -b mitaka mitaka-eol ../nova/venv/bin/pip install -r requirements.txt ../nova/venv/bin/pip install -r test-requirements.txt # dogpile can't cannot import name threading - venv/bin/pip uninstall dogpile.cache dogpile && venv/bin/pip install dogpile.cache venv_mitaka/bin/python -m subunit.run discover ./nova/tests/unit/compute |subunit2pyunit venv_mitaka/bin/python -m subunit.run nova.tests.unit.compute.test_compute |subunit2pyunit venv_mitaka/bin/python -m subunit.run nova.tests.unit.compute.test_compute.ComputeAPITestCase.test_attach_volume |subunit2pyunit venv_mitaka/bin/python -m unittest -v nova.tests.unit.compute.test_compute #make sure the package nova.tests.virt exists venv_mitaka/bin/python -m unittest -v nova.tests.unit.virt.libvirt.test_driver.LibvirtConnTestCase.test_check_can_live_migrate_dest_all_pass_with_over_commit #venv/bin/pep8 venv/bin/flake8

对于horizion的测试:

https://docs.openstack.org/horizon/latest/contributor/testing.html
tox
tox -e py27 -- openstack_dashboard.test.views:DashboardViewsTest.test_urls_ngdetails

charm的unit test:

讯享网# charm unit test sudo pip install virtualenv sudo pip install --upgrade pip proxychains tox -r -epy36 source .tox/py36/bin/activate python -m subunit.run discover -t ./ ./unit_tests |subunit2pyunit python -m subunit.run unit_tests.test_neutron_ovs_context.L3AgentContextTest |subunit2pyunit python -m subunit.run unit_tests.test_neutron_ovs_context.L3AgentContextTest.test_dvr_enabled |subunit2pyunit OS_SUDO_TESTING=True .tox/py36/bin/python -m unittest -v unit_tests.test_neutron_ovs_context.L3AgentContextTest.test_dvr_enabled

更新

今天，发现上面的subunit的方法不work了，　例如运行下面三行报这个错＂TypeError: test_data_port_name() missing 1 required positional argument: ‘config’＂

tox -r -epy38 source .tox/py38/bin/activate python -m subunit.run unit_tests.test_neutron_ovs_context.OVSPluginContextTest.test_data_port_name

估计是现在全面移到python3了，subunit和python3搭配不大好吧，　试了一下，　下面两种方法还可以用：

讯享网OS_SUDO_TESTING=True .tox/py3/bin/python -m unittest -v subunit.run unit_tests.test_neutron_ovs_context.OVSPluginContextTest.test_data_port_name nosetests3 unit_tests/test_neutron_ovs_context.py:OVSPluginContextTest.test_data_port_name

更新

#https://review.opendev.org/#/c// cd nova tox #tox -r -epy38 #source .tox/py38/bin/activate tox -epy38 nova.tests.unit.pci.test_stats.PciDeviceStatsWithTagsTestCase.test_update_device .tox/py38/bin/python -m unittest nova.tests.unit.pci.test_stats.PciDeviceStatsWithTagsTestCase.test_update_device tox -e functional nova.tests.functional.libvirt.test_pci_sriov_servers.SRIOVServersTest.test_create_server_after_change_in_nonsriov_pf_to_sriov_pf .tox/functional/bin/python -m unittest -v nova.tests.functional.libvirt.test_pci_sriov_servers.SRIOVServersTest.test_create_server_after_change_in_nonsriov_pf_to_sriov_pf #NOTE: can use rpdb to debug nova/tests/functional/libvirt/test_pci_sriov_servers.py .tox/functional/bin/pip3 install rpdb import rpdb;rpdb.set_trace() nc 127.0.0.1 4444 python3 -m flake8

unit test怎么写

下列代码是初始为https://bugs.launchpad.net/charm-octavia/+bug/　写的，它是错的因为在删除一个unit时，leader马上会运行下列代码仍然会将正在删除但还未删除的unit弄进去所以是错的。

讯享网+ running_units.append(ch_core.hookenv.local_unit().replace('/', '-')) + for u in ch_core.hookenv.iter_units_for_relation_name('cluster'): + running_units.append(u.unit.replace('/', '-'))

但是这个代码刚好可以演示unit test怎么写。

From 02d8c5185afc6aebfa03d58aa1fe90be4b0d200f Mon Sep 17 00:00:00 2001 From: xxxx Date: Fri, 23 Apr 2021 18:10:32 +0800 Subject: [PATCH] Delete hm prt on unit removal Juju has no pre hook to clean up hm port resources first before removing one octavia unit. This patch will do it in another way, the leader unit finds all running units by querying 'cluster' relations, then finds all mapping between IPv6 management address and unit name by querying DB, finally deletes hm ports for missing units. Closes-Bug:  Change-Id: I88c61b8d2d0b573df7df071ed7978e83b6803c5c --- src/lib/charm/openstack/api_crud.py | 26 ++++++++++++++++++- src/reactive/octavia_handlers.py | 26 ++++++++++++++++++- .../test_lib_charm_openstack_api_crud.py | 25 +++++++++++++++--- unit_tests/test_octavia_handlers.py | 16 +++++++----- 4 files changed, 81 insertions(+), 12 deletions(-) diff --git a/src/lib/charm/openstack/api_crud.py b/src/lib/charm/openstack/api_crud.py index 81c8b31..a0209b8  --- a/src/lib/charm/openstack/api_crud.py +++ b/src/lib/charm/openstack/api_crud.py @@ -251,6 +251,26 @@ def lookup_hm_port(nc, local_unit_name): return +def delete_hm_port(identity_service, local_unit_name): + """Delete port object for Octavia hm port for local unit. + + :param nc: Neutron Client object + :type nc: neutron_client.Client + :param local_unit_name: Name of juju unit, used to build tag name for port + :type local_unit_name: str + :returns: None + :raises: DuplicateResource or any exceptions raised by Keystone and Neutron + clients. + """ + session = session_from_identity_service(identity_service) + try: + nc = init_neutron_client(session) + port = lookup_hm_port(nc, local_unit_name) + nc.delete_port(port['id']) + except NEUTRON_TEMP_EXCS as e: + raise APIUnavailable('neutron', 'ports', e) + + def get_hm_port(identity_service, local_unit_name, local_unit_address, host_id=None): """Get or create a per unit Neutron port for Octavia Health Manager. @@ -507,12 +527,16 @@ def get_port_ips(identity_service): except NEUTRON_TEMP_EXCS as e: raise APIUnavailable('neutron', 'ports', e) + neutron_ip_unit_map = dict() neutron_ip_list = [] for port in resp['ports']: for ip_info in port['fixed_ips']: neutron_ip_list.append(ip_info['ip_address']) + unitname = port['name'].replace( + 'octavia-health-manager-', '').replace('-listen-port', '') + neutron_ip_unit_map[unitname] = ip_info['ip_address'] - return neutron_ip_list + return neutron_ip_list, neutron_ip_unit_map def get_mgmt_network(identity_service, create=True): diff --git a/src/reactive/octavia_handlers.py b/src/reactive/octavia_handlers.py index c1f57d2..cbb7d67  --- a/src/reactive/octavia_handlers.py +++ b/src/reactive/octavia_handlers.py @@ -179,7 +179,9 @@ def update_controller_ip_port_list(): leader_ip_list = leadership.leader_get('controller-ip-port-list') or [] try: - neutron_ip_list = sorted(api_crud.get_port_ips(identity_service)) + neutron_ip_list, neutron_ip_unit_map = api_crud.get_port_ips( + identity_service) + neutron_ip_list = sorted(neutron_ip_list) except api_crud.APIUnavailable as e: ch_core.hookenv.log('Neutron API not available yet, deferring ' 'port discovery. ("{}")' @@ -187,6 +189,28 @@ def update_controller_ip_port_list(): level=ch_core.hookenv.DEBUG) return if neutron_ip_list != sorted(leader_ip_list): + # delete hm port which may be caused by 'juju remove-unit <octavia>' + missing_units = [] + running_units = [] + db_units = neutron_ip_unit_map.keys() + running_units.append(ch_core.hookenv.local_unit().replace('/', '-')) + for u in ch_core.hookenv.iter_units_for_relation_name('cluster'): + running_units.append(u.unit.replace('/', '-')) + for unit_name in db_units: + if unit_name not in running_units: + missing_units.append(unit_name) + for unit_name in missing_units: + neutron_ip_list.remove(neutron_ip_unit_map.get(unit_name)) + try: + ch_core.hookenv.log('deleting hm port for missing ' + 'unit {}'.format(unit_name)) + api_crud.delete_hm_port(identity_service, unit_name) + except api_crud.APIUnavailable as e: + ch_core.hookenv.log('Neutron API not available yet, deferring ' + 'port discovery. ("{}")' + .format(e), + level=ch_core.hookenv.DEBUG) + return leadership.leader_set( {'controller-ip-port-list': json.dumps(neutron_ip_list)}) diff --git a/unit_tests/test_lib_charm_openstack_api_crud.py b/unit_tests/test_lib_charm_openstack_api_crud.py index cbc5776..5ca2628  --- a/unit_tests/test_lib_charm_openstack_api_crud.py +++ b/unit_tests/test_lib_charm_openstack_api_crud.py @@ -247,6 +247,19 @@ class TestAPICrud(test_utils.PatchHelper): nc.update_port.assert_called_with('fake-port-uuid', {'port': {'admin_state_up': True}}) + def test_delete_hm_port(self): + self.patch_object(api_crud, 'session_from_identity_service') + self.patch_object(api_crud, 'init_neutron_client') + identity_service = mock.MagicMock() + nc = mock.MagicMock() + self.init_neutron_client.return_value = nc + nc.list_ports.return_value = {'ports': [{'id': 'fake-port-uuid'}]} + api_crud.delete_hm_port(identity_service, 'fake-unit-name') + self.init_neutron_client.assert_called_once_with( + self.session_from_identity_service()) + nc.list_ports.asssert_called_with(tags='charm-octavia-fake-unit-name') + nc.delete_port.assert_called_with('fake-port-uuid') + def test_setup_hm_port(self): self.patch('subprocess.check_output', 'check_output') self.patch('subprocess.check_call', 'check_call') @@ -305,14 +318,18 @@ class TestAPICrud(test_utils.PatchHelper): self.init_neutron_client.return_value = nc nc.list_ports.return_value = { 'ports': [ - {'fixed_ips': [{'ip_address': '2001:db8:42::42'}]}, - {'fixed_ips': [{'ip_address': '2001:db8:42::51'}]}, + {'name': 'octavia-health-manager-lb-0-listen-port', + 'fixed_ips': [{'ip_address': '2001:db8:42::1'}]}, + {'name': 'octavia-health-manager-lb-1-listen-port', + 'fixed_ips': [{'ip_address': '2001:db8:42::2'}]}, ], } identity_service = mock.MagicMock() + fake_ip_list = ['2001:db8:42::1', '2001:db8:42::2'] + fake_ip_unit_map = {'lb-0': '2001:db8:42::1', 'lb-1': '2001:db8:42::2'} + fake_return_value = (fake_ip_list, fake_ip_unit_map) self.assertEquals(api_crud.get_port_ips(identity_service), - ['2001:db8:42::42', - '2001:db8:42::51']) + fake_return_value) self.init_neutron_client.assert_called_once_with( self.session_from_identity_service()) diff --git a/unit_tests/test_octavia_handlers.py b/unit_tests/test_octavia_handlers.py index 4e3a3c5..8b2560d  --- a/unit_tests/test_octavia_handlers.py +++ b/unit_tests/test_octavia_handlers.py @@ -171,16 +171,20 @@ class TestOctaviaHandlers(test_utils.PatchHelper): self.patch('charms.leadership.leader_set', 'leader_set') self.patch('charms.leadership.leader_get', 'leader_get') self.patch_object(handlers.api_crud, 'get_port_ips') - self.get_port_ips.return_value = [ - '2001:db8:42::42', - '2001:db8:42::51', - ] + fake_ip_list = ['2001:db8:42::1', '2001:db8:42::2'] + fake_ip_unit_map = {'lb-0': '2001:db8:42::1', 'lb-1': '2001:db8:42::2'} + self.get_port_ips.return_value = [fake_ip_list, fake_ip_unit_map] + self.patch_object( + handlers.ch_core.hookenv, 'iter_units_for_relation_name') + fake_relation_name = mock.MagicMock() + fake_relation_name.unit = 'lb/0' + self.iter_units_for_relation_name.return_value = [fake_relation_name] + self.patch_object(handlers.api_crud, 'delete_hm_port') handlers.update_controller_ip_port_list() self.leader_set.assert_called_once_with( { 'controller-ip-port-list': json.dumps([ - '2001:db8:42::42', - '2001:db8:42::51', + '2001:db8:42::1', ])}) def test_render(self): -- 2.25.1

Zaza functional test

讯享网# octavia测试必须enable SG juju bootstrap stsstack --no-gui --bootstrap-series focal --config use-default-secgroup=true --config network=zhhuabj_port_sec_enabled --config image-stream=daily --config image-metadata-url=http://10.230.19.58/swift/v1/simplestreams/data/ zhhuabj-sg juju model-defaults use-default-secgroup=true network=zhhuabj_port_sec_enabled image-stream=daily image-metadata-url=http://10.230.19.58/swift/v1/simplestreams/data/ juju switch zhhuabj-sg # 必须运行在有juju环境的节点上如bastion, # git clone https://github.com/openstack-charmers/zaza.git cd ~/charms # 不能用reactive源码，必须得使用完整的octavia charm # 因为upload charm /home/ubuntu/charms/octavia for series focal #git clone https://github.com/openstack/charm-octavia.git charm-octavia charm pull octavia cd octavia tox -e func-noop source .tox/func-noop/bin/activate ls tests/bundles/focal-ussuri-ha.yaml ls tests/bundles/overlays/focal-ussuri-ha.yaml.j2 ls tests/tests.yaml export OS_VIP00=10.0.0.122 #because we are using zhhuabj-sg functest-run-suite -b focal-ussuri-ha ＃下面是分步运行的步骤，但似乎不work functest-prepare -m testmodel juju switch testmodel export OS_VIP00=10.0.0.122 #注意：也需要调整bundle中的mem大小，以免quota exceed functest-deploy -m testmodel -b ./tests/bundles/focal-ussuri-ha.yaml source ~/xxx/openstack/novarc functest-configure -m testmodel #run test functest-test -m testmodel functest-destroy -m testmodel 或 juju bootstrap stsstack --no-gui --bootstrap-series focal --config use-default-secgroup=true --config network=zhhuabj_port_sec_enabled --config image-stream=daily --config image-metadata-url=http://10.230.19.58/swift/v1/simplestreams/data/ zhhuabj-sg juju model-defaults use-default-secgroup=true network=zhhuabj_port_sec_enabled image-stream=daily image-metadata-url=http://10.230.19.58/swift/v1/simplestreams/data/

tox -e func-target ceph:focal-ussuri

- 调试pip依赖问题

讯享网sudo add-apt-repository ppa:deadsnakes/ppa sudo apt update sudo apt install python3.9 python3.9-dev python3.9-distutils -y alias python=python3.9 alias python3=python3.9 sudo apt install build-essential -y sudo apt install python3-pip python3-dev python3-nose python3-mock -y sudo apt install python3-setuptools libpython3-dev libssl-dev \ libxml2-dev libxslt-dev libxslt1-dev libpq-dev git git-review \ libffi-dev gettext graphviz libjpeg-dev zlib1g-dev psycopg2-binary -y sudo apt install postgresql-server-dev-all libmysqlclient-dev -y git clone https://opendev.org/openstack/aodh.git -b stable/yoga cd aodh tox -e py39 #sudo dpkg -i --force-overwrite /var/cache/apt/archives/python3-distutils_3.10.4-0ubuntu1_all.deb #sudo apt-get -f install

那是因为https://opendev.org/openstack/requirements/raw/branch/stable/yoga/upper-constraints.txt　中缺少了：

讯享网

setuptools===60.9.3;python_version=='3.9'

所以focal master与focal yoga都使用了setuptools=62.1.0, 而这个62.1.0和testr一起工作没问题，但和non-testr一起工作则有问题．刚好master分支才开始使用testr (https://opendev.org/openstack/aodh/commit/63c3466fc3d5f99e307f8784abeeb1ef78).
所以这个focal master没这个问题，而focal yoga则有这个问题．
鉴于修改openstack requirement工程估计不可行，所以提了一个patch来修改aodh自己的requirement.txt (https://review.opendev.org/c/openstack/aodh/+/)

讯享网tox -e generate .tox/generate/bin/generate-constraints -b blacklist.txt -r global-requirements.txt -p python3.6 -p python3.7 -p python3.8 -p python3.9 '>' upper-constraints.txt

其他测试手段：

wget https://releases.openstack.org/constraints/upper/yoga -O yoga.txt .tox/py39/bin/pip install -r yoga.txt .tox/py39/bin/pip install -r requirements.txt .tox/py39/bin/pip install -r test-requirements.txt sudo apt install subunit -y .tox/py39/bin/python -m subunit.run discover -t ./ ./aodh/tests/unit |subunit2pyunit # run-tests.sh sudo apt install python3-os-testr -y export OS_TEST_PATH=aodh/tests/unit .tox/py39/bin/python setup.py testr --slowest --testr-args="--subunit $*" | subunit-trace -f

也试过二分，这个问题与二分无关：

讯享网二分法：https://zhhuabj.blog.csdn.net/article/details/ git log --tags --simplify-by-decoration --pretty="format:%ci %d" git log --since=2015-01-27 --before=2015-05-04 --oneline src/qemu/qemu_migration.c |wc -l # 13.0.0 is xena, 14.0.0 is yoga git bisect start git bisect bad 14.0.0 $ git bisect good 13.0.0 Bisecting: 5 revisions left to test after this (roughly 3 steps) [9fa5ad045b41f90a1f80aca5d8a46a5abbd848c9] Merge "Introduce Guru Meditation Reports into Aodh" $ git bisect bad Bisecting: 2 revisions left to test after this (roughly 1 step) [0d6c43811dc1b531c38bf1fd9c4d345f30ac16b5] Add Python3 yoga unit tests

- pip依赖问题

见： https://github.com/openstack-charmers/zaza-openstack-tests/issues/1165

charm-nova-cloud-controller用了zaza.openstack工程，zaza.openstack工程用了python-ironicclient工程，python-ironicclient工程又用了openstacksdk>=0.18.0, 这会进而用到platformdirs>=3, 而python3.6上没有platformdirs>=3. 1, Only ussuri needs to test against py36, to support bionic-ussuri. victoria, wallaby, xena+ are focal only (py38). 2, 可以像（https://review.opendev.org/c/openstack/charm-hacluster/+//15/test-requirements.txt）一样在charm-nova-cloud-controller的test-requirement.txt中的zaza.openstack前添加下列内容解决 # New openstacksdk versions depend on platformdirs>=3 which does not support # python 3.6 openstacksdk<1.6.0; python_version <= '3.6'

- 本地运行horizon CI测试

讯享网$ grep -r 'envlist' tox.ini envlist = pep8,py39,releasenotes,npm,py3-dj42

原来是可以只用tox而不用zuul本地测的。

sudo apt install python3.11 -y tox -r -epy311 tox -e py3.11 openstack_dashboard/dashboards/identity/projects/tests.py::DetailProjectViewTests::test_detail_view_users_tab

但是当运行’tox -r -epy311’时报了下列错：

讯享网ERROR: could not install deps [-chttps://releases.openstack.org/constraints/upper/master, -r/bak/openstack/horizon/test-requirements.txt, -r/bak/openstack/horizon/requirements.txt, hacking>=3.0.1,<3.1.0]; v = InvocationError("/bak/openstack/horizon/.tox/py311/bin/python -m pip install -chttps://releases.openstack.org/constraints/upper/master -r/bak/openstack/horizon/test-requirements.txt -r/bak/openstack/horizon/requirements.txt 'hacking>=3.0.1,<3.1.0'", 1)

继续运行下列命令也报错：

source .tox/py311/bin/activate pip install --upgrade -r requirements.txt pip install --upgrade -r test-requirements.txt

tox里虽然没有py310，但系统里有python3.10，直接用py310测试就行了。由于有danjo框架还只能通过tox测试，通过pytest (它是基于pytest写的）和nose都不行。

讯享网tox -epy310 openstack_dashboard/dashboards/identity/projects/tests.py::DetailProjectViewTests::test_detail_view_users_tab

另外，重要一点就是，它可能因为使用了pytest，用rpdb死活不行。那样就是：horizon主代码用rpdb(import rpdb;rpdb.set_trace())测试，horizon unit test代码用pdb(import pdb;pdb.set_trace())测试即可。

- 本地运行horizon intergration ci 测试

本地运行horion intgeration ci 测试需要先安装devstack - https://blog.csdn.net/quqi99/article/details/
然后修改一个文件来使用devstack openstack - openstack_dashboard/test/integration_tests/horizon.conf to point the remote devstack (eg: passsword, admin_password, network_cidr, dashboard_url)
之后运行’tox -e integration’就可以本地运行horizon integration ci 测试了，但此时会遇到了一个问题（selenium.common.exceptions.InvalidArgumentException: Message: binary is not a Firefox executable ）它是因为firefox是snap安装的会遇到一个bug (https://github.com/SeleniumHQ/selenium/issues/13252), seleminm是需要使用firefox_bin

$ cat test.py #!/usr/bin/env python # coding=utf-8 import logging import selenium.webdriver import selenium.webdriver.firefox.service logging.basicConfig() logging.getLogger().setLevel(logging.DEBUG) firefox_bin = "/snap/firefox/current/usr/lib/firefox/firefox" #firefoxdriver_bin = "/snap/firefox/current/usr/lib/firefox/geckodriver" options = selenium.webdriver.firefox.options.Options() options.add_argument('--headless') options.binary_location = firefox_bin #service = selenium.webdriver.firefox.service.Service(executable_path=firefoxdriver_bin) service = selenium.webdriver.firefox.service.Service() browser = selenium.webdriver.Firefox(service=service, options=options)

在horizon中可以使用下列patch来传入firefox_bin

讯享网diff --git a/horizon/test/firefox_binary.py b/horizon/test/firefox_binary.py index 34cf7f50f..38adc561f  --- a/horizon/test/firefox_binary.py +++ b/horizon/test/firefox_binary.py @@ -80,10 +80,11 @@ class WebDriver(firefox.webdriver.WebDriver): # Connection refused error happens randomly in integration tests. # When a connection refused exception is raised from start_session # called from WebDriver.__init__, retry __init__. + firefoxBinary = FirefoxBinary(firefox_binary) for i in range(self.CONNREFUSED_RETRY_COUNT + 1): try: super().__init__( - firefox_profile, FirefoxBinary(), timeout, + firefox_profile, firefoxBinary, timeout, desired_capabilities, proxy) if i > 0: # i==0 is normal behavior without connection refused. diff --git a/horizon/test/webdriver.py b/horizon/test/webdriver.py index 5c57f7482..144f5ced6  --- a/horizon/test/webdriver.py +++ b/horizon/test/webdriver.py @@ -24,6 +24,8 @@ from selenium.webdriver.common import by from selenium.webdriver.common import desired_capabilities as dc from selenium.webdriver.remote import webelement +from horizon.test.firefox_binary import FirefoxBinary + # Select the WebDriver to use based on the --selenium-phantomjs switch. if os.environ.get('SELENIUM_PHANTOMJS'): from selenium.webdriver import PhantomJS as WebDriver @@ -113,6 +115,10 @@ class WebElementWrapper(WrapperFindOverride, webelement.WebElement): class WebDriverWrapper(WrapperFindOverride, WebDriver): + + def __init__(self, kwargs): + super().__init__(kwargs) + """Wrapper for webdriver to return WebElementWrapper on find_element.""" def reload_request(self, locator, index): try: diff --git a/openstack_dashboard/test/integration_tests/helpers.py b/openstack_dashboard/test/integration_tests/helpers.py index 0e2ef5fde..a1  --- a/openstack_dashboard/test/integration_tests/helpers.py +++ b/openstack_dashboard/test/integration_tests/helpers.py @@ -14,6 +14,7 @@ import contextlib import io import logging import os +import platform import shutil import socket import subprocess @@ -174,7 +175,14 @@ class BaseTestCase(testtools.TestCase): # Start the Selenium webdriver and setup configuration. desired_capabilities = dict(webdriver.desired_capabilities) desired_capabilities['loggingPrefs'] = {'browser': 'ALL'} + firefox_binary = None + firefox_binary_snap = "/snap/firefox/current/usr/lib/firefox/firefox" + if platform.system().lower() == 'linux': + with open("/proc/mounts", "r") as f: + if any("snap" in line and "firefox" in line for line in f): + firefox_binary = firefox_binary_snap self.driver = webdriver.WebDriverWrapper( + firefox_binary=firefox_binary, desired_capabilities=desired_capabilities ) if self.CONFIG.selenium.maximize_browser:

也可以不用这么麻烦，直接将snap (我报了一个bug , 有时间可以贡献它 - https://bugs.launchpad.net/horizon/+bug/)换成debian包来安装firefox即可。

#https://support.mozilla.org/en-US/kb/install-firefox-linux#w_install-firefox-deb-package-for-debian-based-distributions sudo install -d -m 0755 /etc/apt/keyrings wget -q https://packages.mozilla.org/apt/repo-signing-key.gpg -O- | sudo tee /etc/apt/keyrings/packages.mozilla.org.asc > /dev/null echo "deb [signed-by=/etc/apt/keyrings/packages.mozilla.org.asc] https://packages.mozilla.org/apt mozilla main" | sudo tee -a /etc/apt/sources.list.d/mozilla.list > /dev/null echo ' Package: * Pin: origin packages.mozilla.org Pin-Priority: 1000 ' | sudo tee /etc/apt/preferences.d/mozilla sudo apt-get update && sudo apt-get install firefox #These tests requires geckodriver installed. It could be downloaded from https://github.com/mozilla/geckodriver/releases. wget https://github.com/mozilla/geckodriver/releases/download/v0.34.0/geckodriver-v0.34.0-linux64.tar.gz sudo cp /tmp/geckodriver /usr/local/bin/

运行测试时修改修改tox.ini文件可缩小时间：

讯享网-  pytest --ds=openstack_dashboard.test.settings -v -x --junitxml="{toxinidir}/test_reports/integration_test_results.xml" --html="{toxinidir}/test_reports/integration_test_results.html" --self-contained-html {posargs:{toxinidir}/openstack_dashboard/test/integration_tests}
+  pytest --ds=openstack_dashboard.test.settings -v -x --junitxml="{toxinidir}/test_reports/integration_test_results.xml" --html="{toxinidir}/test_reports/integration_test_results.html" --self-contained-html {posargs:{toxinidir}/openstack_dashboard/test/integration_tests/tests/test_images.py}

但现在发现一个问题，我之前的devstack虽然使用了’TARGET_BRANCH=stable/2023.1’来安装stable/2023.1, 但是devstack的代码却用了master，所以devstack代码里现在的cirros版本是0.6.2不是我们想要的0.5.3, 另外devsatck的local.conf里用了下列两句配置，而我们这次要测的intergration ci测试却是关于openstack_dashboard/test/integration_tests/tests/test_images.py所以我们得将它去掉了再重新安装 devstack

DOWNLOAD_DEFAULT_IMAGES=False IMAGE_URLS="http://download.cirros-cloud.net/0.6.2/cirros-0.6.2-x86_64-disk.img"

但selenium是基于浏览器的模拟点击实现的，如果出错了该怎么调试了。在horizon中可以设置FFMPEG_INSTALLED=True来激活ffmpeg的视频录制来调试。

讯享网# https://review.opendev.org/c/openstack/horizon/+/ #echo $DISPLAY #sudo ffmpeg -video_size 1921x1080 -framerate 15 -f x11grab -i localhost:10.0 output.mp4 export FFMPEG_INSTALLED=True ls ./openstack_dashboard/test/integration_tests/test_reports/TestImagesBasicAngular.test_images_pagination/video.mp4

也会产生一些日志, 里面有selenium调用rest api的记录：

$ find . -name 'test.log' ./openstack_dashboard/test/integration_tests/test_reports/TestDownloadRCFile.test_download_rc_v3_file/test.log ./openstack_dashboard/test/integration_tests/test_reports/TestImagesBasicAngular.test_image_create_delete_from_local_file/test.log ...