Jason Gunthorpe 4db52602a6
vfio: Extend the device migration protocol with PRE_COPY ...
The optional PRE_COPY states open the saving data transfer FD before
reaching STOP_COPY and allows the device to dirty track internal state
changes with the general idea to reduce the volume of data transferred
in the STOP_COPY stage.
While in PRE_COPY the device remains RUNNING, but the saving FD is open.
Only if the device also supports RUNNING_P2P can it support PRE_COPY_P2P,
which halts P2P transfers while continuing the saving FD.
PRE_COPY, with P2P support, requires the driver to implement 7 new arcs
and exists as an optional FSM branch between RUNNING and STOP_COPY:
RUNNING -> PRE_COPY -> PRE_COPY_P2P -> STOP_COPY
A new ioctl VFIO_MIG_GET_PRECOPY_INFO is provided to allow userspace to
query the progress of the precopy operation in the driver with the idea it
will judge to move to STOP_COPY at least once the initial data set is
transferred, and possibly after the dirty size has shrunk appropriately.
This ioctl is valid only in PRE_COPY states and kernel driver should
return -EINVAL from any other migration state.
Compared to the v1 clarification, STOP_COPY -> PRE_COPY is blocked
and to be defined in future.
We also split the pending_bytes report into the initial and sustaining
values, e.g.: initial_bytes and dirty_bytes.
initial_bytes: Amount of initial precopy data.
dirty_bytes: Device state changes relative to data previously retrieved.
These fields are not required to have any bearing to STOP_COPY phase.
It is recommended to leave PRE_COPY for STOP_COPY only after the
initial_bytes field reaches zero. Leaving PRE_COPY earlier might make
things slower.
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com >
Signed-off-by: Shay Drory <shayd@nvidia.com >
Reviewed-by: Kevin Tian <kevin.tian@intel.com >
Reviewed-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com >
Signed-off-by: Yishai Hadas <yishaih@nvidia.com >
Link: https://lore.kernel.org/r/20221206083438.37807-3-yishaih@nvidia.com
Signed-off-by: Alex Williamson <alex.williamson@redhat.com >
2022-12-06 12:36:43 -07:00
2022-06-27 16:16:30 +02:00
2021-12-21 15:09:46 -08:00
2022-09-15 09:08:09 +02:00
2021-09-13 14:51:10 -05:00
2021-08-23 09:49:09 -07:00
2022-09-21 18:42:55 +01:00
2021-01-25 18:44:44 +01:00
2022-06-28 21:26:05 +02:00
2022-09-21 10:44:56 +02:00
2022-06-28 21:26:05 +02:00
2022-06-28 21:26:05 +02:00
2022-06-28 21:26:05 +02:00
2022-08-09 19:51:05 +02:00
2021-10-02 15:50:45 -04:00
2022-06-28 21:26:05 +02:00
2022-04-19 13:13:47 +01:00
2021-06-16 17:47:53 +02:00
2022-09-27 07:48:01 -07:00
2022-08-04 11:41:28 -07:00
2022-04-29 14:38:03 -07:00
2021-10-05 16:14:10 +02:00
2022-05-13 10:56:10 +02:00
2021-11-01 13:36:08 +00:00
2022-08-11 10:31:19 -07:00
2022-06-03 20:09:27 +08:00
2021-05-19 11:40:15 +02:00
2021-02-06 09:22:10 +01:00
2021-02-06 09:22:10 +01:00
2021-02-15 18:28:30 +01:00
2021-04-02 11:18:30 -06:00
2022-06-28 21:26:05 +02:00
2022-10-03 17:44:18 -07:00
2022-06-07 10:20:42 -07:00
2022-09-26 12:28:01 +02:00
2022-09-26 12:28:00 +02:00
2022-05-18 00:11:18 +08:00
2022-09-15 09:08:09 +02:00
2021-07-12 15:17:47 -05:00
2021-01-12 17:31:14 +01:00
2022-05-15 18:31:28 -06:00
2022-09-24 08:50:04 +02:00
2022-09-24 08:50:04 +02:00
2021-11-26 16:48:59 +01:00
2022-06-28 21:26:05 +02:00
2020-12-15 12:13:36 -08:00
2022-09-30 14:32:35 +02:00
2022-04-22 16:12:04 -07:00
2022-01-26 14:49:46 +01:00
2022-06-28 21:26:05 +02:00
2022-07-28 21:56:53 -07:00
2021-10-18 12:28:53 -07:00
2022-08-23 14:54:54 -05:00
2022-08-02 19:50:47 -07:00
2022-06-28 21:26:05 +02:00
2022-06-20 18:21:25 +02:00
2021-03-18 17:03:54 +01:00
2022-08-29 15:32:13 +02:00
2022-06-03 20:09:27 +08:00
2022-07-28 18:05:24 +02:00
2022-10-03 17:33:57 -07:00
2022-10-03 17:33:57 -07:00
2022-08-05 04:17:57 -07:00
2022-08-02 19:50:47 -07:00
2022-04-04 08:55:23 +02:00
2021-04-20 08:59:03 -06:00
2022-06-28 21:26:05 +02:00
2022-06-28 21:26:05 +02:00
2022-06-28 21:26:05 +02:00
2022-06-10 16:40:18 +08:00
2022-02-21 19:38:17 +10:30
2021-01-27 15:13:52 +01:00
2022-02-17 09:09:37 +01:00
2021-02-07 14:51:19 -08:00
2022-09-24 07:00:00 +02:00
2021-10-07 13:51:11 +02:00
2022-08-10 13:49:50 +01:00
2021-02-08 13:01:24 +01:00
2022-05-04 11:06:13 +02:00
2022-03-11 08:28:05 -08:00
2022-09-20 11:53:32 +01:00
2022-02-18 13:08:18 +00:00
2021-11-15 07:53:10 -08:00
2021-11-15 07:53:10 -08:00
2021-06-03 15:31:34 -07:00
2021-03-30 13:29:39 -07:00
2022-07-21 18:10:43 +05:30
2022-02-18 21:20:06 -08:00
2022-06-28 21:26:05 +02:00
2022-06-28 21:26:05 +02:00
2021-07-29 15:06:50 +01:00
2021-05-07 00:26:34 -07:00
2022-03-17 16:49:57 -07:00
2022-09-15 09:08:09 +02:00
2021-03-10 12:45:16 -08:00
2022-10-03 07:52:13 +01:00
2022-08-18 20:37:35 -07:00
2022-06-28 21:26:05 +02:00
2022-09-23 12:02:03 +01:00
2022-03-11 08:28:27 -08:00
2022-06-28 21:26:05 +02:00
2021-07-21 08:14:33 -07:00
2022-09-20 09:13:38 +02:00
2022-06-28 21:26:05 +02:00
2022-06-28 21:26:05 +02:00
2022-09-28 18:31:37 -07:00
2022-07-17 07:52:46 -07:00
2022-09-30 07:50:59 -06:00
2021-07-21 08:14:33 -07:00
2022-02-04 20:24:45 -08:00
2021-07-21 08:14:33 -07:00
2022-02-28 13:25:48 +01:00
2021-08-18 07:23:15 -06:00
2022-06-28 21:26:05 +02:00
2022-06-28 21:26:05 +02:00
2021-10-05 06:54:16 -05:00
2022-05-31 11:36:57 +02:00
2022-06-28 21:26:05 +02:00
2022-06-28 21:26:05 +02:00
2022-06-28 21:26:05 +02:00
2020-12-15 12:13:36 -08:00
2022-06-03 20:09:27 +08:00
2022-07-12 16:50:44 -04:00
2022-01-11 15:44:27 -05:00
2021-06-17 14:25:39 -04:00
2022-09-29 10:23:08 +01:00
2022-09-20 09:13:38 +02:00
2022-09-29 18:43:04 +02:00
2022-05-25 09:51:36 -07:00
2022-07-08 10:46:53 -07:00
2022-05-10 06:30:05 -06:00
2022-08-29 10:44:08 +02:00
2022-08-02 12:34:03 -04:00
2021-03-10 09:34:06 +01:00
2021-03-24 08:26:29 +01:00
2021-10-21 23:36:28 +02:00
2022-02-09 12:00:11 +00:00
2022-05-01 17:45:35 +01:00
2022-06-20 21:15:04 +02:00
2022-04-24 08:27:37 +01:00
2021-09-03 09:58:17 -07:00
2022-06-28 21:26:05 +02:00
2022-01-11 18:45:02 -08:00
2021-07-26 14:45:18 +02:00
2022-06-28 20:45:43 -07:00
2022-02-19 16:05:54 +00:00
2021-01-23 12:38:42 -08:00
2021-06-30 15:34:04 -06:00
2022-06-28 21:26:05 +02:00
2022-06-30 13:14:35 +02:00
2022-06-28 21:26:05 +02:00
2021-12-29 12:31:36 -08:00
2021-06-18 13:02:45 -07:00
2022-09-07 16:46:03 +02:00
2022-09-29 18:39:39 -07:00
2021-03-11 16:12:59 -08:00
2021-12-27 14:58:37 +00:00
2021-01-25 09:36:25 -05:00
2022-02-25 18:50:12 -05:00
2022-03-13 12:59:35 -04:00
2021-09-14 11:11:20 +02:00
2021-10-21 17:25:17 +02:00
2022-09-03 17:04:29 +02:00
2022-05-20 06:17:33 -06:00
2022-02-27 10:58:04 -08:00
2021-01-04 11:42:26 -05:00
2022-08-26 19:31:20 -07:00
2022-07-19 15:38:04 -07:00
2021-07-06 10:37:46 -05:00
2022-10-27 10:27:32 +02:00
2021-12-23 09:32:51 +01:00
2021-12-27 17:12:58 +01:00
2022-09-20 09:13:38 +02:00
2022-09-29 18:52:05 -07:00
2020-12-21 17:36:48 -08:00
2022-04-22 18:50:54 +01:00
2021-03-25 15:31:22 -07:00
2022-09-28 22:38:07 +02:00
2022-03-18 09:44:19 -05:00
2022-06-28 21:26:05 +02:00
2022-02-17 09:09:38 +01:00
2021-12-10 17:10:55 -08:00
2022-05-09 14:00:07 +02:00
2022-09-24 08:31:54 +02:00
2022-06-28 21:26:05 +02:00
2021-01-25 15:14:16 -08:00
2022-03-13 11:49:53 -05:00
2022-02-02 13:11:34 +01:00
2022-03-23 19:58:40 +01:00
2022-06-28 21:26:05 +02:00
2022-06-28 21:26:05 +02:00
2022-05-03 14:11:58 -07:00
2022-08-22 07:52:51 -06:00
2022-08-03 16:29:08 -07:00
2022-09-20 12:33:22 +02:00
2022-06-28 21:26:05 +02:00
2022-07-16 08:16:01 +02:00
2022-06-27 14:41:31 +02:00
2022-06-27 14:44:20 +02:00
2022-04-07 16:47:12 +02:00
2021-05-18 16:20:54 -05:00
2022-02-02 07:42:41 -08:00
2022-07-18 11:19:17 +01:00
2022-07-11 19:48:33 -07:00
2022-06-01 16:48:05 -07:00
2022-01-08 17:41:00 +09:00
2022-09-11 19:47:04 -05:00
2022-03-31 13:04:44 -07:00
2022-06-28 21:26:05 +02:00
2022-06-16 19:58:21 -07:00
2022-06-30 13:14:35 +02:00
2022-06-28 21:26:05 +02:00
2022-06-01 15:55:25 -07:00
2021-02-11 18:25:05 -08:00
2022-04-26 10:17:03 +02:00
2021-01-07 16:17:32 +01:00
2022-02-03 19:50:49 +01:00
2022-04-06 13:48:02 +01:00
2022-09-27 17:29:09 -07:00
2021-04-10 10:36:34 +02:00
2022-06-27 16:25:41 +02:00
2022-05-25 10:08:59 -07:00
2022-09-23 19:09:56 -06:00
2022-06-28 21:26:05 +02:00
2022-08-31 09:07:53 +02:00
2022-09-11 20:25:48 -07:00
2022-01-20 08:52:54 +02:00
2021-03-22 10:24:07 +01:00
2022-08-30 07:32:30 +02:00
2021-03-22 10:22:22 +01:00
2022-10-07 20:00:42 -04:00
2022-08-11 04:26:08 -04:00
2022-07-11 09:54:37 +02:00
2022-12-06 12:36:43 -07:00
2022-08-12 09:50:34 -07:00
2022-08-11 04:31:15 -04:00
2022-10-25 16:40:07 +01:00
2022-06-28 21:26:05 +02:00
2022-10-07 09:32:41 -04:00
2021-04-08 12:26:34 +02:00
2022-08-11 04:06:40 -04:00
2022-03-28 16:52:58 -04:00
2021-11-04 16:36:54 +01:00
2021-09-29 09:22:29 +02:00
2021-11-01 05:26:48 -04:00
2022-05-10 07:22:28 -04:00
2021-12-06 15:03:05 +01:00
2021-11-10 15:32:38 +01:00
2022-08-30 22:44:25 -07:00
2022-08-11 04:06:40 -04:00
2021-08-26 22:28:03 +02:00
2022-08-16 01:40:24 -04:00
2021-08-05 10:24:08 +01:00
2021-03-07 09:07:16 +01:00
2021-09-05 16:23:09 -04:00
2021-10-08 16:21:53 +01:00
2021-06-12 13:16:45 -07:00
2022-08-24 12:51:50 +01:00
4db52602a6