[Linaro-validation] [Lava-announce] Migration to V2

[Jakub Pavelek]

Hi Neil/guys

thanks for the update. In your presentation you talk about pre-requisites
for the V2 migration and one of them is "Make all devices on all instances
usable with V2". When will this happen for Juno, HiKey and X15?

I'm guessing that LMG would need few months on top of that milestone to
migrate our test jobs, re-establish dashboards/trends, setup pre-merge
testing, etc before step #1 (lava-dispatcher will *disable* V1 submissions
entirely) happens.

Everyone using LAVA should be aware of V1 and V2 and that V1 is going
to be turned off. The plan for how to do that has been discussed
within the team and within Linaro. The most recent presentation on the
http://connect.linaro.org/resource/las16/las16-503/
(Note: the video will auto-start, be aware.)
http://www.slideshare.net/linaroorg/las16503-lava-v2-migration
The dates for each step in the migration are not finalised but will
occur during 2017, this is advance notice.
0: During 2017, regular announcements will be made on this list before
each major change. In addition, the regular release announcements will
include details of the major changes in that release.
1: The first major change will be that a new release of lava-server
and lava-dispatcher will *disable* V1 submissions entirely. Any queued
V1 test jobs will become invalid upon this upgrade,
2: A later release will remove V1 documentation and start removing the
V1 source code.
3: A release will be made which *forcibly deletes V1 test jobs,
bundles, filters and image reports*. This is required to complete the
removal of the V1 source code.
4: The final step of the migration is a release containing no V1
source code at all. Only V2 support will remain. This is essential as
the V1 source code is currently blocking some useful additions to V2
as well as plans for future development. The data cannot remain
accessible without the V1 source code. There is not and can not be any
support for converting V1 data to V2.
Please take this chance to read up on the V2 documentation, including
how Results, Queries and Charts replace the deprecated V1 support from
filters and image reports.
Work has started upstream on making sure the migration runs smoothly,
especially when deleting large numbers of V1 test cases.
Remember: Once the first major step is released, upgrades will stop
running any V1 test jobs. Subsequent releases will include changes
which force the deletion of your V1 test data.
If you want to preserve your V1 data, consider setting up a read-only
reference instance which is then pinned at a particular release of
LAVA to prevent deletion of the V1 data.
--
Neil Williams
=============
http://www.linux.codehelp.co.uk/
_______________________________________________
Lava-announce mailing list
https://lists.linaro.org/mailman/listinfo/lava-announce

[Neil Williams]

On Thu, 15 Dec 2016 10:49:19 +0100

Post by Jakub Pavelek
Hi Neil/guys
thanks for the update. In your presentation you talk about
pre-requisites for the V2 migration and one of them is "Make all
devices on all instances usable with V2". When will this happen for
Juno, HiKey and X15?

HiKey V2 support using LXC is now available on staging but there are
lab changes required to allow HiKeys in production to support V2. Any
one HiKey board cannot support V1 and V2 simultaneously due to
limitations of the HiKey hardware. Tests using Android and LXC are
running on staging to support migrating the test definitions away from
MultiNode to V2 using LXC:
https://staging.validation.linaro.org/results/157649

It's up to the lab team to manage the upgrade of hikey devices in
production to V2 support, dependent on expanding the initial test jobs
on staging to the full range of tests required for LMG.

Juno Android support is in development.

X15 support in LAVA V2 has not been requested and has hardware
requirements which need lab work. I'm anticipating that X15 support is
desired for LAVA V2 but no work has yet been done on V2 support for X15.

Post by Jakub Pavelek
I'm guessing that LMG would need few months on top of that milestone
to migrate our test jobs, re-establish dashboards/trends, setup
pre-merge testing, etc before step #1 (lava-dispatcher will *disable*
V1 submissions entirely) happens.

As shown in the slides and agreed at LAS16, 3 months V2 data will be
required for all teams before the decision is made to disable V1
submissions. You can listen to the presentation again if you want to
hear the discussion, using the link provided.

Post by Jakub Pavelek
http://connect.linaro.org/resource/las16/las16-503/
(Note: the video will auto-start, be aware.)

[Jakub Pavelek]

Hi,

Since HiKey is the only board (that we care about) supported in V2 right
now, I was wondering if we should ask for few (1-5) NEW HiKey boards to be
deployed as V2 only instances in staging. That would allow us to start
migration work without waiting for Juno/X15, get some hands-on with
ImageReports2 replacements, etc.

Also - how is it with the database? Is there some dev/playground database
that we could submit results to during migration so that we do not pollute
database with errors and mistakes?

Post by Neil Williams
On Thu, 15 Dec 2016 10:49:19 +0100

Post by Jakub Pavelek
Hi Neil/guys
thanks for the update. In your presentation you talk about
pre-requisites for the V2 migration and one of them is "Make all
devices on all instances usable with V2". When will this happen for
Juno, HiKey and X15?

HiKey V2 support using LXC is now available on staging but there are
lab changes required to allow HiKeys in production to support V2. Any
one HiKey board cannot support V1 and V2 simultaneously due to
limitations of the HiKey hardware. Tests using Android and LXC are
running on staging to support migrating the test definitions away from
https://staging.validation.linaro.org/results/157649
It's up to the lab team to manage the upgrade of hikey devices in
production to V2 support, dependent on expanding the initial test jobs
on staging to the full range of tests required for LMG.
Juno Android support is in development.
X15 support in LAVA V2 has not been requested and has hardware
requirements which need lab work. I'm anticipating that X15 support is
desired for LAVA V2 but no work has yet been done on V2 support for X15.

Post by Jakub Pavelek
I'm guessing that LMG would need few months on top of that milestone
to migrate our test jobs, re-establish dashboards/trends, setup
pre-merge testing, etc before step #1 (lava-dispatcher will *disable*
V1 submissions entirely) happens.

As shown in the slides and agreed at LAS16, 3 months V2 data will be
required for all teams before the decision is made to disable V1
submissions. You can listen to the presentation again if you want to
hear the discussion, using the link provided.

Post by Jakub Pavelek
http://connect.linaro.org/resource/las16/las16-503/
(Note: the video will auto-start, be aware.)

--
Neil Williams
=============
http://www.linux.codehelp.co.uk/

[Neil Williams]

On Mon, 19 Dec 2016 11:12:13 +0000

On Mon, 19 Dec 2016 10:35:58 +0100

Post by Jakub Pavelek
Hi,
Since HiKey is the only board (that we care about) supported in V2
right now, I was wondering if we should ask for few (1-5) NEW HiKey
boards to be deployed as V2 only instances in staging.

That's a question for the lab team, there are hardware implications
beyond just the extra HiKey boards due to the extra Cambrionix hub
support.

Post by Jakub Pavelek
That would
allow us to start migration work without waiting for Juno/X15, get
some hands-on with ImageReports2 replacements, etc.

The 1 HiKey on staging is not particularly busy and LXC HiKey V2 test
jobs are quicker than V1. This can be further improved by not
repeating the same tests every time when all that's happening is
development of the test definition and supporting scripts.
However, there are issues with how adb is used within these test
definitions. https://github.com/mwasilew/AndroidViewClient.git has
issues with certain builds of adb. I have results from this test job
https://staging.validation.linaro.org/scheduler/job/157649/definition
Re-submissions of this testjob have failed due to a newer version of
adb becoming available inside the LXC which caused the view client to
fail.
Note also that even if you want to use a custom-built adb during the
test shell (which is supported), adb and fastboot must still exist in
the packages: list to the LXC as these binaries are essential to
deploying the test images onto the HiKey in the first place.
Any feedback on improving the documentation of how to develop test
jobs based on HiKey, Android and LXC will be appreciated but the core
of how LXC works, what it does and how it differs from a full VM is
beyond the scope of the LAVA documentation. (The docs do include
links for further reading on LXC.)

Post by Jakub Pavelek
Also - how is it with the database? Is there some dev/playground
database that we could submit results to during migration so that we
do not pollute database with errors and mistakes?

V2 supports omitting specific results from queries and charts but the
simplest way to handle this is to use the metadata support to mark the
V2 jobs as "in development" and then match for or against that piece
of metadata in queries based on the V2 results. Deleting data is the
wrong option here.
staging.validation.linaro.org is not intended to provide CI for teams
other than the LAVA software team. However it is useful in this
complex situation where the V1 requirements are so different to the V2
support. Once a set of V2 test definitions have been developed, the
Lab team can prepare to add / divide the HiKey devices in production
to support release testing using V2.
Please use staging as a way to develop the V2 test definitions - once
a test job is working, work can move on to another definition until
there is enough coverage to allocate HiKey devices in production to V2
only. It is vital during this process that only one element is changed
at a time, so stick to the one build (I've been using 238) and re-test
all V2 definitions when changing either the build or the LXC support
or the scripts which are common to multiple test jobs.

The files from build 238 are preserved here:
http://images.validation.linaro.org/functional-test-images/hikey/
(To cover for when snapshots eventually removes those files.)