GH-48593: [C++] C++20: use standard calendar / timezone APIs

[rok]

Rationale for this change

Switch to std::chrono for MSVC to be able to use the system-provided timezone automatically on Windows.

What changes are included in this PR?

This adds chrono_internal.h that uses C++20 std::chrono timezone/calendar APIs on compilers with support (MSVC only for now) and falls back to vendored date.h otherwise.

Are these changes tested?

Partially tested locally and partially to be tested on CI.

Are there any user-facing changes?

Yes, Windows users will no longer need to install the IANA tzdb (see instructions here and here). We possibly have tzdb download set up in CI too and should update it appropriately.

• GitHub Issue: [C++] C++20: use standard calendar / timezone APIs #48593

[github-actions]

⚠️ GitHub issue #48593 has been automatically assigned in GitHub to PR creator.

[rok]

It seems that std::chrono on GCC (14.3.0, 15.2.0) potentially has a bug that triggers some of our tests. Meanwhile std::chrono on MSVC 19.44 ( 14.44) appears to be pass them and is correct or at least consistent with vendored date.h. I would therefore advise we only switch to std::chrono on MSVC for now as that gives us the most benefit anyway (users no longer have to deal with the tz db).

Below is the explanation and reproduction of the bug.

// GCC libstdc++ DST bug reproduction
//
// The AN to AS Transition Bug
// ---------------------------
// Source https://github.com/eggert/tz/blob/c37fbc3249c1a1334948b38f3bca47dee5c11dd1/australasia#L165-L192
// Australia/Broken_Hill used the AN (New South Wales) rules until 2000, then
// switched to AS (South Australia) rules. Under AN rules, DST started the last
// Sunday of October and ended the last Sunday of March. For the 1999-2000
// summer, DST started October 31, 1999 and would end March 26, 2000. February
// 29, 2000 falls squarely within this DST period, so the correct offset should
// be 9:30 base + 1:00 DST = 10:30 (630 minutes).
//
// Why GCC's Data is Wrong
// -----------------------
// When libstdc++ processes the zone transition from AN rules to AS rules (which
// happens in year 2000), it appears to lose or reset the DST state inherited
// from the AN rules. Instead of recognizing that DST is still active from the
// October 1999 transition, it reports offset=570 (just the 9:30 base) with
// save=0. The inconsistency is evident: it returns abbrev="ACDT" (daylight
// time) but the offset and save values indicate standard time. The AN rules
// clearly show DST should be active until the last Sunday of March 2000.
//
// Compile: g++ -std=c++20 -o gcc_dst_bug gcc_libstdcxx_dst_bug.cpp
// Expected: 630 (10:30 = 9:30 base + 1:00 DST)
// Actual:   570 (9:30 = base only, DST missing)

#include <chrono>
#include <iostream>

int main() {
  using namespace std::chrono;
  auto* tz = locate_zone("Australia/Broken_Hill");
  auto info = tz->get_info(sys_days{2000y / February / 29d} + 23h + 23min + 23s);
  std::cout << duration_cast<minutes>(info.offset).count() << "\n";
}

[rok]

@pitrou

[pitrou]

It seems that std::chrono on GCC (14.3.0, 15.2.0) potentially has a bug that triggers some of our tests.

Is the bug reported somewhere? If not, can you do that?

[pitrou]

We possibly have tzdb download set up in CI too and should update it appropriately.

Yes, I think we should do so. There are also a bunch of code snippets in the C++ and Python codebase that could be removed, IIRC.

[rok]

Is the bug reported somewhere? If not, can you do that?

It seems to be related to a known issue, I added comment explaining our case.

[pitrou]

+1, this is looking very good now. @raulcd Do you want to make a final pass over this?

[raulcd]

@github-actions crossbow submit -g cpp

[github-actions]

Revision: 32fbc78

Submitted crossbow builds: ursacomputing/crossbow @ actions-1a8d03ac7f

Task	Status
example-cpp-minimal-build-static
example-cpp-minimal-build-static-system-dependency
example-cpp-tutorial
test-build-cpp-fuzz
test-conda-cpp
test-conda-cpp-valgrind
test-debian-13-cpp-amd64
test-debian-13-cpp-i386
test-debian-experimental-cpp-gcc-15
test-fedora-42-cpp
test-ubuntu-22.04-cpp
test-ubuntu-22.04-cpp-20
test-ubuntu-22.04-cpp-bundled
test-ubuntu-22.04-cpp-emscripten
test-ubuntu-22.04-cpp-no-threading
test-ubuntu-24.04-cpp
test-ubuntu-24.04-cpp-bundled-offline
test-ubuntu-24.04-cpp-gcc-13-bundled
test-ubuntu-24.04-cpp-gcc-14
test-ubuntu-24.04-cpp-minimal-with-formats
test-ubuntu-24.04-cpp-thread-sanitizer

[raulcd]

The changes look good overall to me. I think there's more than the gcc bug on some of these issues found, see my other review comment.
Nonetheless I am unsure how can we fight the timezone database nuances on those where they are not consistent, historical timezone changes, etcetera, where there's no agreement between different timezone databases.

[raulcd]

I am confused by this comment, on that specific job failure we use:

-- The C compiler identification is MSVC 19.44.35222.0
-- The CXX compiler identification is MSVC 19.44.35222.0

not GCC. Isn't this related to differences between historical timezones depending on the timezone database used?

[rok]

Changing here and elsewhere to latest.

[rok]

Thanks for review @raulcd! I pushed a couple of changes since. Nothing major, but still. I'll do another pass tomorrow to make sure all TODOs point to the right issues.

[jorisvandenbossche]

In a comment on the docs, you mentioned keeping the note about ORC and having to set TZDIR, but then here it is apparently not needed for our tests?

[rok]

I suspect we don't really test for this as I believe ORC still needs TZDIR, see #48601 (review). I'll do another check tomorrow.

[h-vetinari]

If your testing uses conda environments, then orc will pick up the tzdata from the environment: apache/orc#1882