Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes video recorder ffmpeg on Centos7 and RHEL7 #1893

Merged
merged 3 commits into from
Jul 27, 2021

Conversation

zlig
Copy link
Contributor

@zlig zlig commented Apr 24, 2020

  • Conditionally select different arguments for ffmpeg
  • Adds meaningful error from the third party application in charge of encoding

…ul error for third party application in charge of encoding
@christopherhesse
Copy link
Contributor

Should we instead depend on imageio-ffmpeg to get a consistent binary across different platforms? We could use imageio_ffmpeg.get_ffmpeg_exe() instead of relying on system ffmpeg. @pzhokhov

@pzhokhov
Copy link
Collaborator

yeah, I like imageio-ffmpeg python package path better too

@zlig
Copy link
Contributor Author

zlig commented Apr 25, 2020

Are you suggesting to replace distutils.spawn.find_executable() by imageio_ffmpeg.get_ffmpeg_exe(), such as the following?

250         if "avconv" in os.path.split(imageio_ffmpeg.get_ffmpeg_exe())[-1]:
251             self.backend = 'avconv'
252         elif "ffmpeg" in os.path.split(imageio_ffmpeg.get_ffmpeg_exe())[-1]:
253             self.backend = 'ffmpeg'
254         else:
255             raise error.DependencyNotInstalled("""Found neither the ffmpeg nor avconv executables. On OS X, you can install ffmpeg via `brew install ffmpeg`. On most Ubuntu variants, `sudo apt-get install ff    mpeg` should do it. On Ubuntu 14.04, however, you'll need to install avconv with `sudo apt-get install libav-tools`.""")

Not sure if it is an improvement question readability, but having new dependencies in the requirements for imageio and imageio_ffmpeg will probably resolve other install issues.

@christopherhesse
Copy link
Contributor

Sort of, I think we could forget "avconv" entirely, add imageio_ffmpeg to the dependencies, and then always assume that imageio's ffmpeg works and accepts the options we are giving it. Does that package work fine on centos?

@vwxyzjn
Copy link
Contributor

vwxyzjn commented Sep 1, 2021

Hey following up with this issue. @zlig could you create a Dockerfile based on Centos7 or RHEL7? This would greatly help us understand and test the PR.

zlig added a commit to zlig/gym that referenced this pull request Sep 6, 2021
* Conditionally select different arguments for ffmpeg, and add meaningful error for third party application in charge of encoding

* Consistency with other logger() calls

Co-authored-by: J K Terry <[email protected]>
@zlig
Copy link
Contributor Author

zlig commented Sep 6, 2021

Hi @vwxyzjn

Actually, the RHEL/CentOS 7 systems aren't supported anymore.

Provided the following CentOS 8 Dockerfile into the PR 2400, if there is an issue number you need me to attach against. It is still work-in-progress as I'm still getting 15x failing tests at the moment...

# A Dockerfile based on CentOS 8 that sets up a full Gym install with test dependencies
FROM centos:8
ARG PYTHON_VERSION
ENV PYTHON_VERSION=${PYTHON_VERSION:-36}

# Install pre-requisites
RUN dnf clean all 
RUN dnf install -y epel-release
RUN dnf install -y --enablerepo=powertools wget unzip cmake gcc patchelf \
                python$PYTHON_VERSION python$PYTHON_VERSION-devel python3-pyopengl freeglut-devel \
                mesa-libOSMesa-devel mesa-libOSMesa mesa-libGL-devel glfw-devel xorg-x11-server-Xvfb
RUN rm -rf /var/cache/dnf

# Download and install mujoco
RUN mkdir /root/.mujoco && \
    cd /root/.mujoco  && \
    curl -O https://www.roboti.us/download/mujoco200_linux.zip && \
    unzip mujoco200_linux.zip && \
    mv /root/.mujoco/mujoco200_linux/ /root/.mujoco/mujoco200/ && \
    echo DUMMY_KEY > /root/.mujoco/mjkey.txt

ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/root/.mujoco/mujoco200/bin

COPY . /usr/local/gym/
WORKDIR /usr/local/gym/

# Set Python 3 as default interpreter
RUN ln -s /usr/bin/python3 /usr/bin/python
RUN ln -s /usr/bin/pip3 /usr/bin/pip
RUN python --version

# Install required Python modules
RUN pip install pyglet && pip install ffmpeg && pip install -U 'mujoco-py<2.1,>=2.0' && pip install -r test_requirements.txt

# Execution
ENTRYPOINT ["/usr/local/gym/bin/docker_entrypoint"]

Also, just for reference, this is a CentOS 7 Dockerfile, but I need to spend a bit more time on it :

# A Dockerfile that sets up a full Gym install with test dependencies
ARG PYTHON_VERSION
FROM centos:7
#RUN apt-get -y update && apt-get install -y unzip libglu1-mesa-dev libgl1-mesa-dev libosmesa6-dev xvfb patchelf ffmpeg cmake swig
RUN yum clean all 
RUN yum install -y epel-release
RUN yum install -y wget unzip cmake gcc 
RUN yum install -y python36 python36-devel
RUN yum install -y glfw-devel mesa-libOSMesa-devel mesa-libGL-devel patchelf python-xvfbwrapper

# Download mujoco
RUN mkdir /root/.mujoco && \
    cd /root/.mujoco  && \
    curl -O https://www.roboti.us/download/mjpro150_linux.zip && \
    unzip mjpro150_linux.zip && \
    echo DUMMY_KEY > /root/.mujoco/mjkey.txt

ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/root/.mujoco/mjpro150/bin

COPY . /usr/local/gym/
WORKDIR /usr/local/gym/

RUN echo 'alias python="python3"' >> ~/.bashrc
RUN echo 'alias pip="pip3"' >> ~/.bashrc
RUN source ~/.bashrc
RUN python --version

RUN pip3 install ffmpeg && pip3 install -U 'mujoco-py<1.50.2,>=1.50.1' && pip3 install -r test_requirements.txt

ENTRYPOINT ["/usr/local/gym/bin/docker_entrypoint"]

zlig added a commit to zlig/gym that referenced this pull request Oct 1, 2021
jkterry1 pushed a commit that referenced this pull request Oct 4, 2021
* Reverts some changes from #1893, and propose to install imageio-ffmpeg instead

* Reformat file as required by the Python Black linter
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants