[Refactor] Refactor fileio #2543

xiexinch · 2023-01-31T11:58:05Z

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

Use the new fileio from mmengine
open-mmlab/mmengine#533

Modification

Use mmengine.fileio to repalce FileClient in mmseg/datasets
Use mmengine.fileio to repalce FileClient in mmseg/datasets/transforms
Use mmengine.fileio to repalce FileClient in mmseg/visualization

BC-breaking (Optional)

TODO

Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.

Checklist

Pre-commit or other linting tools are used to fix the potential lint issues.
The modification is covered by complete unit tests. If not, please add more unit test to ensure the correctness.
If the modification has potential influence on downstream projects, this PR should be tested with downstream projects, like MMDet or MMDet3D.
The documentation has been modified accordingly, like docstring or example tutorials.

MeowZheng

please remove the modifications of projects and add the version of mmcv and mmengine that backend_args need.

MeowZheng · 2023-02-01T02:31:21Z

mmseg/__init__.py

    f'MMEngine=={mmengine.__version__} is used but incompatible. ' \
    f'Please install mmengine>={mmengine_min_version}, '\
-    f'<{mmengine_max_version}.'
+    f'<={mmengine_max_version}.'


Suggested change

f'<={mmengine_max_version}.'

f'<{mmengine_max_version}.'

MeowZheng · 2023-02-01T02:31:30Z

mmseg/__init__.py


 mmengine_min_version = digit_version(MMENGINE_MIN)
 mmengine_max_version = digit_version(MMENGINE_MAX)
 mmengine_version = digit_version(mmengine.__version__)

-assert (mmengine_min_version <= mmengine_version < mmengine_max_version), \
+assert (mmengine_min_version <= mmengine_version <= mmengine_max_version), \


Suggested change

assert (mmengine_min_version <= mmengine_version <= mmengine_max_version), \

assert (mmengine_min_version <= mmengine_version < mmengine_max_version), \

mmseg/datasets/transforms/loading.py

codecov · 2023-02-01T02:39:18Z

Codecov Report

Base: 83.40% // Head: 83.35% // Decreases project coverage by -0.06% ⚠️

Coverage data is based on head (6608185) compared to base (18ee41e).
Patch coverage: 88.00% of modified lines in pull request are covered.

Additional details and impacted files

@@             Coverage Diff             @@
##           dev-1.x    #2543      +/-   ##
===========================================
- Coverage    83.40%   83.35%   -0.06%     
===========================================
  Files          145      145              
  Lines         8510     8505       -5     
  Branches      1274     1273       -1     
===========================================
- Hits          7098     7089       -9     
- Misses        1198     1202       +4     
  Partials       214      214

Flag	Coverage Δ
unittests	`83.35% <88.00%> (-0.06%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
mmseg/datasets/transforms/loading.py	`81.25% <72.72%> (-4.36%)`	⬇️
mmseg/__init__.py	`92.10% <100.00%> (ø)`
mmseg/datasets/basesegdataset.py	`100.00% <100.00%> (ø)`
mmseg/datasets/isaid.py	`100.00% <100.00%> (ø)`
mmseg/datasets/voc.py	`100.00% <100.00%> (ø)`
mmseg/engine/hooks/visualization_hook.py	`97.05% <100.00%> (-0.24%)`	⬇️
mmseg/datasets/transforms/transforms.py	`90.53% <0.00%> (+0.12%)`	⬆️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

MeowZheng · 2023-02-01T09:06:38Z

docs/en/notes/faq.md

@@ -4,37 +4,19 @@ We list some common troubles faced by many users and their corresponding solutio

 ## Installation

-The compatible MMSegmentation and MMCV versions are as below. Please install the correct version of MMCV to avoid installation issues.
+The compatible MMSegmentation, MMCV and MMEngine versions are as below. Please install the correct version of MMCV to avoid installation issues.


Suggested change

The compatible MMSegmentation, MMCV and MMEngine versions are as below. Please install the correct version of MMCV to avoid installation issues.

The compatible MMSegmentation, MMCV and MMEngine versions are as below. Please install the correct versions of them to avoid installation issues.

MeowZheng · 2023-02-01T09:09:17Z

docs/en/notes/faq.md

+| MMSegmentation version |          MMCV version          | MMEngine version  | MMClassification (optional) version | MMDetection (optional) version |
+| :--------------------: | :----------------------------: | :---------------: | :---------------------------------: | :----------------------------: |
+|     dev-1.x branch     |        mmcv >= 2.0.0rc4        | MMEngine >= 0.2.0 |           mmcls>=1.0.0rc0           | mmdet>=3.0.0rc4, \<=3.0.0rc5>  |
+|       1.x branch       |        mmcv >= 2.0.0rc4        | MMEngine >= 0.2.0 |           mmcls>=1.0.0rc0           | mmdet>=3.0.0rc4, \<=3.0.0rc5>  |


as 1.x branch updates when mmseg release, I suggest modifying 1.x branch dependency in bump pr

get

MeowZheng · 2023-02-01T09:09:59Z

mmseg/__init__.py

+MMCV_MAX = '2.1.0'
 MMENGINE_MIN = '0.1.0'


Suggested change

MMCV_MAX = '2.1.0'

MMENGINE_MIN = '0.1.0'

MMCV_MAX = '2.1.0'

MMENGINE_MIN = '0.2.0'

MeowZheng · 2023-02-01T09:11:19Z

requirements/mminstall.txt

@@ -1,4 +1,4 @@
 mmcls>=1.0.0rc0
-mmcv==2.0.0rc3
+mmcv>=2.0.0rc4
 mmdet==3.0.0rc5
 mmengine>=0.1.0,<1.0.0


Suggested change

mmengine>=0.1.0,<1.0.0

mmengine>=0.2.0,<1.0.0

* [Model offload] Add nice warning * Treat sequential and model offload differently. Sequential raises an error because the operation would fail with a cryptic warning later. * Forcibly move to cpu when offloading. * make style * one more fix * make fix-copies * up --------- Co-authored-by: Pedro Cuenca <[email protected]>

## Motivation Use the new fileio from mmengine open-mmlab/mmengine#533 ## Modification 1. Use `mmengine.fileio` to repalce FileClient in mmseg/datasets 2. Use `mmengine.fileio` to repalce FileClient in mmseg/datasets/transforms 3. Use `mmengine.fileio` to repalce FileClient in mmseg/visualization ## BC-breaking (Optional) we modify all the dataset configurations, so please use the latest config file.

xiexinch added 3 commits January 31, 2023 17:32

support train

8eb655a

fix ut

191bb11

update mmcv limit

9a41e8b

xiexinch added WIP Work in process 1.x Related issue of 1.x version labels Jan 31, 2023

mm-assistant bot assigned xiexinch Jan 31, 2023

update ci

f7124d5

MeowZheng reviewed Jan 31, 2023

View reviewed changes

remove modifications from projects

cd2d34e

xiexinch removed the WIP Work in process label Feb 1, 2023

MeowZheng approved these changes Feb 1, 2023

View reviewed changes

xiexinch added 3 commits February 1, 2023 10:53

update docstring

2ce0340

minor change

46f28cf

update requirements and faq.md

6053e6c

MengzhangLI approved these changes Feb 1, 2023

View reviewed changes

xiexinch added 2 commits February 1, 2023 16:16

mmcv max version

8f2fe13

update faq document

f20c990

MeowZheng reviewed Feb 1, 2023

View reviewed changes

xiexinch added 2 commits February 1, 2023 17:13

update mmengine limit

f5c4d75

update faq

6608185

MeowZheng approved these changes Feb 1, 2023

View reviewed changes

MeowZheng merged commit 124b87c into open-mmlab:dev-1.x Feb 1, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Refactor] Refactor fileio #2543

[Refactor] Refactor fileio #2543

xiexinch commented Jan 31, 2023

MeowZheng left a comment

MeowZheng Feb 1, 2023

MeowZheng Feb 1, 2023

codecov bot commented Feb 1, 2023 •

edited

Loading

MeowZheng Feb 1, 2023

MeowZheng Feb 1, 2023

xiexinch Feb 1, 2023

MeowZheng Feb 1, 2023

MeowZheng Feb 1, 2023

	assert (mmengine_min_version <= mmengine_version <= mmengine_max_version), \
	assert (mmengine_min_version <= mmengine_version < mmengine_max_version), \

	The compatible MMSegmentation, MMCV and MMEngine versions are as below. Please install the correct version of MMCV to avoid installation issues.
	The compatible MMSegmentation, MMCV and MMEngine versions are as below. Please install the correct versions of them to avoid installation issues.

	\| 1.x branch \| mmcv >= 2.0.0rc4 \| MMEngine >= 0.2.0 \| mmcls>=1.0.0rc0 \| mmdet>=3.0.0rc4, \<=3.0.0rc5> \|
	\| 1.x branch \| mmcv == 2.0.0rc3 \| MMEngine >= 0.1.0 \| mmcls>=1.0.0rc0 \| mmdet>=3.0.0rc4, \<=3.0.0rc5> \|

[Refactor] Refactor fileio #2543

[Refactor] Refactor fileio #2543

Conversation

xiexinch commented Jan 31, 2023

Motivation

Modification

BC-breaking (Optional)

Use cases (Optional)

Checklist

MeowZheng left a comment

Choose a reason for hiding this comment

MeowZheng Feb 1, 2023

Choose a reason for hiding this comment

MeowZheng Feb 1, 2023

Choose a reason for hiding this comment

codecov bot commented Feb 1, 2023 • edited Loading

Codecov Report

MeowZheng Feb 1, 2023

Choose a reason for hiding this comment

MeowZheng Feb 1, 2023

Choose a reason for hiding this comment

xiexinch Feb 1, 2023

Choose a reason for hiding this comment

MeowZheng Feb 1, 2023

Choose a reason for hiding this comment

MeowZheng Feb 1, 2023

Choose a reason for hiding this comment

codecov bot commented Feb 1, 2023 •

edited

Loading