Tests fail locally, but pass on travis

[ikhsan]

I'm currently trying to add functionality to slather to output coverage files as static htmls. I've already added the basic functionality in my branch and currently trying to add tests to cover those up.

Although, I had a problem when running the tests in my local machine. I'm currently running Mac OS X 10.10.3, ruby 2.0.0p481 and Xcode 6.3.1. What I did was:

• Clone repo
• bundle install
• bundle exec rspec

Firstly, I have a warning from iPhone simulator saying that the service could not established. Here is the warning message:
xcodebuild[3176:118116] [MT] iPhoneSimulator: SimVerifier returned: Error Domain=NSPOSIXErrorDomain Code=53 "Simulator verification failed." UserInfo=0x7f91a1d6e280 {NSLocalizedFailureReason=A connection to the simulator verification service could not be established., NSLocalizedRecoverySuggestion=Ensure that Xcode.app is installed on a volume with ownership enabled., NSLocalizedDescription=Simulator verification failed.}

Secondly, I have 3 tests failing, which all related to the fixture Branches.m on line 13. It seems that somehow the generated outputs differ from the fixture files. Below are my rspec failures:

  1) Slather::CoverageService::CoberturaXmlOutput#post should create an XML report spanning all coverage files
     Failure/Error: EquivalentXml.equivalent?(current_xml_doc, fixture_xml_doc).should be_truthy
       expected: truthy value
            got: false
     # ./spec/slather/coverage_service/cobertura_xml_spec.rb:38:in `block (3 levels) in <top (required)>'

  2) Slather::CoverageService::GutterJsonOutput#post should print out the coverage for each file, and then total coverage
     Failure/Error: expect(current_json).to be_json_eql(fixture_json)
       Expected equivalent JSON
       Diff:

       @@ -185,10 +185,9 @@
                "short_text": "-"
              },
              {
       -        "background_color": "0x35CC4B",
                "line": 13,
                "long_text": "",
       -        "short_text": "2"
       +        "short_text": "-"
              },
              {
                "line": 14,

       @@ -432,10 +431,9 @@
                "short_text": "-"
              },
              {
       -        "background_color": "0xFC635E",
                "line": 13,
                "long_text": "",
       -        "short_text": "#####"
       +        "short_text": "-"
              },
              {
                "line": 14,
     # ./spec/slather/coverage_service/gutter_json_spec.rb:26:in `block (3 levels) in <top (required)>'

  3) Slather::CoverageService::SimpleOutput#post should print out the coverage for each file, and then total coverage
     Failure/Error: fixtures_project.post
       #<Slather::Project> path:`/Users/ikhsan/Open Source/slather/spec/fixtures/fixtures.xcodeproj` UUID:`8C9A2025195965F00013B6B3` received :puts with unexpected arguments
         expected: ("spec/fixtures/fixtures/fixtures.m: 2 of 4 lines (50.00%)")
              got: ("spec/fixtures/fixtures/more_files/Branches.m: 9 of 19 lines (47.37%)")
     # ./lib/slather/coverage_service/simple_output.rb:23:in `block in post'
     # ./lib/slather/coverage_service/simple_output.rb:13:in `each'
     # ./lib/slather/coverage_service/simple_output.rb:13:in `post'
     # ./spec/slather/coverage_service/simple_output_spec.rb:32:in `block (3 levels) in <top (required)>'

In contradiction, I pushed the same commit to run it on travis. It works fine. I'd really appreciate it if you guys could shed some light on how to run the tests properly.

Many thanks for this wonderful library! 👍

[marklarr]

@orta and @ixnixnixn -- try this out and let me know if it fixes your issue. http://stackoverflow.com/questions/24224193/xcodebuild-test-not-running-xctestcase-unit-test-file/25276814#25276814

courtesy of @mattt here

[ikhsan]

The problem still persists for me. I think the simulator warning is more minor than the discrepancies that caused the test failures in the local environment.

Anyways, I've managed to make #76 pass by adjusting the fixture files.

[marklarr]

What did you adjust in the fixtures to get it working locally?

[ikhsan]

There are 2 discrepancies between my local test and travis' that causes the failures. Apparently my local tests doesn't detect :

• Branches.m's line 13
• peekaview.m's line 13

The fix was by altering the numbers and html attributes in the html fixtures; bed99fb, df5e464 and 875cc09. So it still fails locally, but passes on travis.

[marklarr]

Cool! Can we close this out, then?

[ikhsan]

Yeah, I'm happy to close this issue.