folly::Subprocess Class Reference

#include <Subprocess.h>

Classes
struct	ChildPipe
struct	DangerousPostForkPreExecCallback
class	Options
struct	Pipe
class	ReadLinesCallback

Public Types
using	FdCallback = folly::Function< bool(int, int)>

Public Member Functions
	Subprocess (const Subprocess &)=delete
Subprocess &	operator= (const Subprocess &)=delete
	Subprocess (Subprocess &&)=default
Subprocess &	operator= (Subprocess &&)=default
	Subprocess ()
	Subprocess (const std::vector< std::string > &argv, const Options &options=Options(), const char *executable=nullptr, const std::vector< std::string > *env=nullptr)
	~Subprocess ()
	Subprocess (const std::string &cmd, const Options &options=Options(), const std::vector< std::string > *env=nullptr)
pid_t	pid () const
ProcessReturnCode	returnCode () const
ProcessReturnCode	poll (struct rusage *ru=nullptr)
bool	pollChecked ()
ProcessReturnCode	wait ()
void	waitChecked ()
void	sendSignal (int signal)
void	terminate ()
void	kill ()
std::pair< IOBufQueue, IOBufQueue >	communicateIOBuf (IOBufQueue input=IOBufQueue())
std::pair< std::string, std::string >	communicate (StringPiece input=StringPiece())
void	communicate (FdCallback readCallback, FdCallback writeCallback)
void	enableNotifications (int childFd, bool enabled)
bool	notificationsEnabled (int childFd) const
void	closeParentFd (int childFd)
void	setAllNonBlocking ()
int	parentFd (int childFd) const
int	stdinFd () const
int	stdoutFd () const
int	stderrFd () const
std::vector< ChildPipe >	takeOwnershipOfPipes ()

Static Public Member Functions
template<class Callback >
static auto	readLinesCallback (Callback &&fdLineCb, uint64_t maxLineLength=0, char delimiter= '\n', uint64_t bufSize=1024) -> ReadLinesCallback< typename std::decay< Callback >::type >

Static Public Attributes
static const int	CLOSE = -1
static const int	PIPE = -2
static const int	PIPE_IN = -3
static const int	PIPE_OUT = -4

Private Member Functions
void	spawn (std::unique_ptr< const char *[]> argv, const char *executable, const Options &options, const std::vector< std::string > *env)
void	spawnInternal (std::unique_ptr< const char *[]> argv, const char *executable, Options &options, const std::vector< std::string > *env, int errFd)
int	prepareChild (const Options &options, const sigset_t *sigmask, const char *childDir) const
int	runChild (const char *executable, char **argv, char **env, const Options &options) const
void	readChildErrorPipe (int pfd, const char *executable)
size_t	findByChildFd (const int childFd) const

Private Attributes
pid_t	pid_ {-1}
ProcessReturnCode	returnCode_
std::vector< Pipe >	pipes_

Detailed Description

Subprocess.

Definition at line 269 of file Subprocess.h.

Member Typedef Documentation

using folly::Subprocess::FdCallback = folly::Function<bool(int, int)>

Communicate with the child until all pipes to/from the child are closed.

== Semantics ==

readCallback(pfd, cfd) will be called whenever there's data available on any pipe from the child (PIPE_OUT). pfd is the file descriptor in the parent (that you use to read from); cfd is the file descriptor in the child (used for identifying the stream; 1 = child's standard output, 2 = child's standard error, etc)

writeCallback(pfd, cfd) will be called whenever a pipe to the child is writable (PIPE_IN). pfd is the file descriptor in the parent (that you use to write to); cfd is the file descriptor in the child (used for identifying the stream; 0 = child's standard input, etc)

The read and write callbacks must read from / write to pfd and return false during normal operation. Return true to tell communicate() to close the pipe. For readCallback, this might send SIGPIPE to the child, or make its writes fail with EPIPE, so you should generally avoid returning true unless you've reached end-of-file.

communicate() returns when all pipes to/from the child are closed; the child might stay alive after that, so you must still wait(). Conversely, the child may quit long before its pipes are closed, since its descendants can keep them alive forever.

Most users won't need to use this callback version; the simpler version of communicate (which buffers data in memory) will probably work fine.

== Things you must get correct ==

1) You MUST consume all data passed to readCallback (or return true to close the pipe). Similarly, you MUST write to a writable pipe (or return true to close the pipe). To do otherwise is an error that can result in a deadlock. You must do this even for pipes you are not interested in.

2) pfd is nonblocking, so be prepared for read() / write() to return -1 and set errno to EAGAIN (in which case you should return false). Use readNoInt() from FileUtil.h to handle interrupted reads for you.

3) Your callbacks MUST NOT call any of the Subprocess methods that manipulate the pipe FDs. Check the docblocks, but, for example, neither closeParentFd (return true instead) nor takeOwnershipOfPipes are safe. Stick to reading/writing from pfd, as appropriate.

== Good to know ==

1) See ReadLinesCallback for an easy way to consume the child's output streams line-by-line (or tokenized by another delimiter).

2) "Wait until the descendants close the pipes" is usually the behavior you want, since the descendants may have something to say even if the immediate child is dead. If you need to be able to force-close all parent FDs, communicate() will NOT work for you. Do it your own way by using takeOwnershipOfPipes().

Why not? You can return "true" from your callbacks to sever active pipes, but inactive ones can remain open indefinitely. It is impossible to safely close inactive pipes while another thread is blocked in communicate(). This is BY DESIGN. Racing communicate()'s read/write callbacks can result in wrong I/O and data corruption. This class would need internal synchronization and timeouts, a poor and expensive implementation choice, in order to make closeParentFd() thread-safe.

Definition at line 718 of file Subprocess.h.

Constructor & Destructor Documentation

folly::Subprocess::Subprocess ( const Subprocess &  )

delete

folly::Subprocess::Subprocess ( Subprocess &&  )

default

folly::Subprocess::Subprocess

(

)

Create an uninitialized subprocess.

In this state it can only be destroyed, or assigned to using the move assignment operator.

Definition at line 188 of file Subprocess.cpp.

{}

folly::Subprocess::Subprocess ( const std::vector< std::string > &  argv, const Options &  options = Options(), const char *  executable = nullptr, const std::vector< std::string > *  env = nullptr  )

explicit

Create a subprocess from the given arguments. argv[0] must be listed. If not-null, executable must be the actual executable being used (otherwise it's the same as argv[0]).

If env is not-null, it must contain name=value strings to be used as the child's environment; otherwise, we inherit the environment from the parent. env must be null if options.usePath is set.

Definition at line 190 of file Subprocess.cpp.

                                     {
  if (argv.empty()) {
    throw std::invalid_argument("argv must not be empty");
  }
  if (!executable) {
    executable = argv[0].c_str();
  }
  spawn(cloneStrings(argv), executable, options, env);
}

folly::Subprocess::~Subprocess

(

)

Definition at line 216 of file Subprocess.cpp.

References folly::ProcessReturnCode::RUNNING.

                        {
  CHECK_NE(returnCode_.state(), ProcessReturnCode::RUNNING)
      << "Subprocess destroyed without reaping child";
}

folly::Subprocess::Subprocess ( const std::string &  cmd, const Options &  options = Options(), const std::vector< std::string > *  env = nullptr  )

explicit

Create a subprocess run as a shell command (as shell -c 'command')

The shell to use is taken from the environment variable $SHELL, or /bin/sh if $SHELL is unset.

Definition at line 204 of file Subprocess.cpp.

References argv, and folly::Subprocess::Options::usePath_.

                                     {
  if (options.usePath_) {
    throw std::invalid_argument("usePath() not allowed when running in shell");
  }

  std::vector<std::string> argv = {"/bin/sh", "-c", cmd};
  spawn(cloneStrings(argv), argv[0].c_str(), options, env);
}

Member Function Documentation

void folly::Subprocess::closeParentFd

(

int

childFd

)

Close the parent file descriptor given a file descriptor in the child. DO NOT USE from communicate() callbacks; make them return true instead.

Definition at line 901 of file Subprocess.cpp.

                                          {
  int idx = findByChildFd(childFd);
  pipes_[idx].pipe.close(); // May throw
  pipes_.erase(pipes_.begin() + idx);
}

std::pair< std::string, std::string > folly::Subprocess::communicate

(

StringPiece

input = StringPiece()

)

Definition at line 740 of file Subprocess.cpp.

References folly::Range< Iter >::data(), folly::gen::move, folly::Range< Iter >::size(), and folly::IOBufQueue::wrapBuffer().

Referenced by TEST().

                                                                       {
  IOBufQueue inputQueue;
  inputQueue.wrapBuffer(input.data(), input.size());

  auto outQueues = communicateIOBuf(std::move(inputQueue));
  auto outBufs =
      std::make_pair(outQueues.first.move(), outQueues.second.move());
  std::pair<std::string, std::string> out;
  if (outBufs.first) {
    outBufs.first->coalesce();
    out.first.assign(
        reinterpret_cast<const char*>(outBufs.first->data()),
        outBufs.first->length());
  }
  if (outBufs.second) {
    outBufs.second->coalesce();
    out.second.assign(
        reinterpret_cast<const char*>(outBufs.second->data()),
        outBufs.second->length());
  }
  return out;
}

void folly::Subprocess::communicate	(	FdCallback	readCallback,
		FdCallback	writeCallback
	)

Definition at line 801 of file Subprocess.cpp.

References folly::checkUnixError(), i, folly::netops::poll(), and folly::ProcessReturnCode::RUNNING.

                              {
  // This serves to prevent wait() followed by communicate(), but if you
  // legitimately need that, send a patch to delete this line.
  returnCode_.enforce(ProcessReturnCode::RUNNING);
  setAllNonBlocking();

  std::vector<pollfd> fds;
  fds.reserve(pipes_.size());
  std::vector<size_t> toClose; // indexes into pipes_
  toClose.reserve(pipes_.size());

  while (!pipes_.empty()) {
    fds.clear();
    toClose.clear();

    for (auto& p : pipes_) {
      pollfd pfd;
      pfd.fd = p.pipe.fd();
      // Yes, backwards, PIPE_IN / PIPE_OUT are defined from the
      // child's point of view.
      if (!p.enabled) {
        // Still keeping fd in watched set so we get notified of POLLHUP /
        // POLLERR
        pfd.events = 0;
      } else if (p.direction == PIPE_IN) {
        pfd.events = POLLOUT;
      } else {
        pfd.events = POLLIN;
      }
      fds.push_back(pfd);
    }

    int r;
    do {
      r = ::poll(fds.data(), fds.size(), -1);
    } while (r == -1 && errno == EINTR);
    checkUnixError(r, "poll");

    for (size_t i = 0; i < pipes_.size(); ++i) {
      auto& p = pipes_[i];
      auto parentFd = p.pipe.fd();
      DCHECK_EQ(fds[i].fd, parentFd);
      short events = fds[i].revents;

      bool closed = false;
      if (events & POLLOUT) {
        DCHECK(!(events & POLLIN));
        if (writeCallback(parentFd, p.childFd)) {
          toClose.push_back(i);
          closed = true;
        }
      }

      // Call read callback on POLLHUP, to give it a chance to read (and act
      // on) end of file
      if (events & (POLLIN | POLLHUP)) {
        DCHECK(!(events & POLLOUT));
        if (readCallback(parentFd, p.childFd)) {
          toClose.push_back(i);
          closed = true;
        }
      }

      if ((events & (POLLHUP | POLLERR)) && !closed) {
        toClose.push_back(i);
        closed = true;
      }
    }

    // Close the fds in reverse order so the indexes hold after erase()
    for (int idx : boost::adaptors::reverse(toClose)) {
      auto pos = pipes_.begin() + idx;
      pos->pipe.close(); // Throws on error
      pipes_.erase(pos);
    }
  }
}

std::pair< IOBufQueue, IOBufQueue > folly::Subprocess::communicateIOBuf

(

IOBufQueue

input = IOBufQueue()

)

Communicate with the child until all pipes to/from the child are closed.

The input buffer is written to the process' stdin pipe, and data is read from the stdout and stderr pipes. Non-blocking I/O is performed on all pipes simultaneously to avoid deadlocks.

The stdin pipe will be closed after the full input buffer has been written. An error will be thrown if a non-empty input buffer is supplied but stdin was not configured as a pipe.

Returns a pair of buffers containing the data read from stdout and stderr. If stdout or stderr is not a pipe, an empty IOBuf queue will be returned for the respective buffer.

Note that communicate() and communicateIOBuf() both return when all pipes to/from the child are closed; the child might stay alive after that, so you must still wait().

communicateIOBuf() uses IOBufQueue for buffering (which has the advantage that it won't try to allocate all data at once), but it does store the subprocess's entire output in memory before returning.

communicate() uses strings for simplicity.

Definition at line 763 of file Subprocess.cpp.

References folly::IOBufQueue::empty(), and folly::gen::move.

Referenced by TEST().

                      {
  // If the user supplied a non-empty input buffer, make sure
  // that stdin is a pipe so we can write the data.
  if (!input.empty()) {
    // findByChildFd() will throw std::invalid_argument if no pipe for
    // STDIN_FILENO exists
    findByChildFd(STDIN_FILENO);
  }

  std::pair<IOBufQueue, IOBufQueue> out;

  auto readCallback = [&](int pfd, int cfd) -> bool {
    if (cfd == STDOUT_FILENO) {
      return handleRead(pfd, out.first);
    } else if (cfd == STDERR_FILENO) {
      return handleRead(pfd, out.second);
    } else {
      // Don't close the file descriptor, the child might not like SIGPIPE,
      // just read and throw the data away.
      return discardRead(pfd);
    }
  };

  auto writeCallback = [&](int pfd, int cfd) -> bool {
    if (cfd == STDIN_FILENO) {
      return handleWrite(pfd, input);
    } else {
      // If we don't want to write to this fd, just close it.
      return true;
    }
  };

  communicate(std::move(readCallback), std::move(writeCallback));

  return out;
}

void folly::Subprocess::enableNotifications	(	int	childFd,
		bool	enabled
	)

communicate() callbacks can use this to temporarily enable/disable notifications (callbacks) for a pipe to/from the child. By default, all are enabled. Useful for "chatty" communication – you want to disable write callbacks until you receive the expected message.

Disabling a pipe does not free you from the requirement to consume all incoming data. Failing to do so will easily create deadlock bugs.

Throws if the childFd is not known.

Definition at line 881 of file Subprocess.cpp.

Referenced by TEST().

                                                              {
  pipes_[findByChildFd(childFd)].enabled = enabled;
}

size_t folly::Subprocess::findByChildFd ( const int  childFd ) const

private

Definition at line 889 of file Subprocess.cpp.

References pipe().

                                                  {
  auto pos = std::lower_bound(
      pipes_.begin(), pipes_.end(), childFd, [](const Pipe& pipe, int fd) {
        return pipe.childFd < fd;
      });
  if (pos == pipes_.end() || pos->childFd != childFd) {
    throw std::invalid_argument(
        folly::to<std::string>("child fd not found ", childFd));
  }
  return pos - pipes_.begin();
}

void folly::Subprocess::kill ( )

inline

Definition at line 611 of file Subprocess.h.

              {
    sendSignal(SIGKILL);
  }

bool folly::Subprocess::notificationsEnabled

(

int

childFd

)

const

Are notifications for one pipe to/from child enabled? Throws if the childFd is not known.

Definition at line 885 of file Subprocess.cpp.

                                                       {
  return pipes_[findByChildFd(childFd)].enabled;
}

Subprocess& folly::Subprocess::operator= ( const Subprocess &  )

delete

Subprocess& folly::Subprocess::operator= ( Subprocess &&  )

default

int folly::Subprocess::parentFd ( int  childFd ) const

inline

Get parent file descriptor corresponding to the given file descriptor in the child. Throws if childFd isn't a pipe (PIPE_IN / PIPE_OUT). Do not close() the returned file descriptor; use closeParentFd, above.

Definition at line 869 of file Subprocess.h.

                                  {
    return pipes_[findByChildFd(childFd)].pipe.fd();
  }

pid_t folly::Subprocess::pid

(

)

const

Return the child's pid, or -1 if the child wasn't successfully spawned or has already been wait()ed upon.

Definition at line 674 of file Subprocess.cpp.

References b, folly::checkUnixError(), folly::IOBufQueue::front(), folly::io::detail::CursorBase< Derived, BufType >::peekBytes(), folly::IOBufQueue::postallocate(), folly::IOBufQueue::preallocate(), folly::readNoInt(), folly::IOBufQueue::trimStart(), and folly::writeNoInt().

                            {
  return pid_;
}

ProcessReturnCode folly::Subprocess::poll

(

struct rusage *

ru = nullptr

)

Poll the child's status and return it. Return the exit status if the subprocess had quit, or RUNNING otherwise. Throws an std::logic_error if called on a Subprocess whose status is no longer RUNNING. No other exceptions are possible. Aborts on egregious violations of contract, e.g. if you wait for the underlying process without going through this Subprocess instance.

Definition at line 618 of file Subprocess.cpp.

References folly::ProcessReturnCode::enforce(), folly::ProcessReturnCode::make(), and folly::ProcessReturnCode::RUNNING.

Referenced by runParent().

                                                    {
  returnCode_.enforce(ProcessReturnCode::RUNNING);
  DCHECK_GT(pid_, 0);
  int status;
  pid_t found = ::wait4(pid_, &status, WNOHANG, ru);
  // The spec guarantees that EINTR does not occur with WNOHANG, so the only
  // two remaining errors are ECHILD (other code reaped the child?), or
  // EINVAL (cosmic rays?), both of which merit an abort:
  PCHECK(found != -1) << "waitpid(" << pid_ << ", &status, WNOHANG)";
  if (found != 0) {
    // Though the child process had quit, this call does not close the pipes
    // since its descendants may still be using them.
    returnCode_ = ProcessReturnCode::make(status);
    pid_ = -1;
  }
  return returnCode_;
}

bool folly::Subprocess::pollChecked

(

)

Poll the child's status. If the process is still running, return false. Otherwise, return true if the process exited with status 0 (success), or throw CalledProcessError if the process exited with a non-zero status.

Definition at line 636 of file Subprocess.cpp.

References folly::netops::poll(), and folly::ProcessReturnCode::RUNNING.

                             {
  if (poll().state() == ProcessReturnCode::RUNNING) {
    return false;
  }
  checkStatus(returnCode_);
  return true;
}

FOLLY_POP_WARNING int folly::Subprocess::prepareChild ( const Options &  options, const sigset_t *  sigmask, const char *  childDir  ) const

private

Definition at line 495 of file Subprocess.cpp.

References folly::netops::close(), folly::Subprocess::Options::closeOtherFds_, folly::Subprocess::Options::dangerousPostForkPreExecCallback_, folly::pushmi::operators::error(), folly::Subprocess::Options::fdActions_, folly::Subprocess::Options::processGroupLeader_, and folly::sig.

                                {
  // While all signals are blocked, we must reset their
  // dispositions to default.
  for (int sig = 1; sig < NSIG; ++sig) {
    ::signal(sig, SIG_DFL);
  }

  {
    // Unblock signals; restore signal mask.
    int r = pthread_sigmask(SIG_SETMASK, sigmask, nullptr);
    if (r != 0) {
      return r; // pthread_sigmask() returns an errno value
    }
  }

  // Change the working directory, if one is given
  if (childDir) {
    if (::chdir(childDir) == -1) {
      return errno;
    }
  }

  // We don't have to explicitly close the parent's end of all pipes,
  // as they all have the FD_CLOEXEC flag set and will be closed at
  // exec time.

  // Close all fds that we're supposed to close.
  for (auto& p : options.fdActions_) {
    if (p.second == CLOSE) {
      if (::close(p.first) == -1) {
        return errno;
      }
    } else if (p.second != p.first) {
      if (::dup2(p.second, p.first) == -1) {
        return errno;
      }
    }
  }

  // If requested, close all other file descriptors.  Don't close
  // any fds in options.fdActions_, and don't touch stdin, stdout, stderr.
  // Ignore errors.
  if (options.closeOtherFds_) {
    for (int fd = getdtablesize() - 1; fd >= 3; --fd) {
      if (options.fdActions_.count(fd) == 0) {
        ::close(fd);
      }
    }
  }

#if __linux__
  // Opt to receive signal on parent death, if requested
  if (options.parentDeathSignal_ != 0) {
    const auto parentDeathSignal =
        static_cast<unsigned long>(options.parentDeathSignal_);
    if (prctl(PR_SET_PDEATHSIG, parentDeathSignal, 0, 0, 0) == -1) {
      return errno;
    }
  }
#endif

  if (options.processGroupLeader_) {
    if (setpgrp() == -1) {
      return errno;
    }
  }

  // The user callback comes last, so that the child is otherwise all set up.
  if (options.dangerousPostForkPreExecCallback_) {
    if (int error = (*options.dangerousPostForkPreExecCallback_)()) {
      return error;
    }
  }

  return 0;
}

void folly::Subprocess::readChildErrorPipe ( int  pfd, const char *  executable  )

private

Read from the error pipe, and throw SubprocessSpawnError if the child failed before calling exec().

Definition at line 589 of file Subprocess.cpp.

References deadlock::info(), folly::readNoInt(), folly::SubprocessSpawnError::SubprocessSpawnError(), and folly::detail::distributed_mutex::wait().

                                                                   {
  ChildErrorInfo info;
  auto rc = readNoInt(pfd, &info, sizeof(info));
  if (rc == 0) {
    // No data means the child executed successfully, and the pipe
    // was closed due to the close-on-exec flag being set.
    return;
  } else if (rc != sizeof(ChildErrorInfo)) {
    // An error occurred trying to read from the pipe, or we got a partial read.
    // Neither of these cases should really occur in practice.
    //
    // We can't get any error data from the child in this case, and we don't
    // know if it is successfully running or not.  All we can do is to return
    // normally, as if the child executed successfully.  If something bad
    // happened the caller should at least get a non-normal exit status from
    // the child.
    LOG(ERROR) << "unexpected error trying to read from child error pipe "
               << "rc=" << rc << ", errno=" << errno;
    return;
  }

  // We got error data from the child.  The child should exit immediately in
  // this case, so wait on it to clean up.
  wait();

  // Throw to signal the error
  throw SubprocessSpawnError(executable, info.errCode, info.errnoValue);
}

template<class Callback >

static auto folly::Subprocess::readLinesCallback ( Callback &&  fdLineCb, uint64_t  maxLineLength = 0, char  delimiter = '\n', uint64_t  bufSize = 1024  ) -> ReadLinesCallback<typename std::decay<Callback>::type>

inlinestatic

Definition at line 816 of file Subprocess.h.

                                                            {
    return ReadLinesCallback<typename std::decay<Callback>::type>(
        std::forward<Callback>(fdLineCb), maxLineLength, delimiter, bufSize);
  }

ProcessReturnCode folly::Subprocess::returnCode ( ) const

inline

Return the child's status (as per wait()) if the process has already been waited on, -1 if the process is still running, or -2 if the process hasn't been successfully started. NOTE that this does not call waitpid() or Subprocess::poll(), but simply returns the status stored in the Subprocess object.

Definition at line 569 of file Subprocess.h.

References folly::netops::poll(), and folly::detail::distributed_mutex::wait().

Referenced by TEST().

                                       {
    return returnCode_;
  }

int folly::Subprocess::runChild ( const char *  executable, char **  argv, char **  env, const Options &  options  ) const

private

Definition at line 575 of file Subprocess.cpp.

References folly::Subprocess::Options::usePath_.

                                  {
  // Now, finally, exec.
  if (options.usePath_) {
    ::execvp(executable, argv);
  } else {
    ::execve(executable, argv, env);
  }
  return errno;
}

void folly::Subprocess::sendSignal

(

int

signal

)

Send a signal to the child. Shortcuts for the commonly used Unix signals are below.

Definition at line 668 of file Subprocess.cpp.

References folly::checkUnixError(), and folly::ProcessReturnCode::RUNNING.

                                      {
  returnCode_.enforce(ProcessReturnCode::RUNNING);
  int r = ::kill(pid_, signal);
  checkUnixError(r, "kill");
}

void folly::Subprocess::setAllNonBlocking

(

)

Set all pipes from / to child to be non-blocking. communicate() does this for you.

Definition at line 239 of file Subprocess.cpp.

References folly::checkUnixError().

                                   {
  for (auto& p : pipes_) {
    int fd = p.pipe.fd();
    int flags = ::fcntl(fd, F_GETFL);
    checkUnixError(flags, "fcntl");
    int r = ::fcntl(fd, F_SETFL, flags | O_NONBLOCK);
    checkUnixError(r, "fcntl");
  }
}

void folly::Subprocess::spawn ( std::unique_ptr< const char *[]>  argv, const char *  executable, const Options &  options, const std::vector< std::string > *  env  )

private

Definition at line 249 of file Subprocess.cpp.

References argv, folly::checkUnixError(), folly::netops::close(), folly::Subprocess::Options::detach_, FOLLY_GCC_DISABLE_WARNING, FOLLY_PUSH_WARNING, folly::makeGuard(), folly::gen::move, pipe(), SCOPE_EXIT, folly::Subprocess::Options::usePath_, and folly::detail::distributed_mutex::wait().

                                     {
  if (optionsIn.usePath_ && env) {
    throw std::invalid_argument(
        "usePath() not allowed when overriding environment");
  }

  // Make a copy, we'll mutate options
  Options options(optionsIn);

  // On error, close all pipes_ (ignoring errors, but that seems fine here).
  auto pipesGuard = makeGuard([this] { pipes_.clear(); });

  // Create a pipe to use to receive error information from the child,
  // in case it fails before calling exec()
  int errFds[2];
#if FOLLY_HAVE_PIPE2
  checkUnixError(::pipe2(errFds, O_CLOEXEC), "pipe2");
#else
  checkUnixError(::pipe(errFds), "pipe");
#endif
  SCOPE_EXIT {
    CHECK_ERR(::close(errFds[0]));
    if (errFds[1] >= 0) {
      CHECK_ERR(::close(errFds[1]));
    }
  };

#if !FOLLY_HAVE_PIPE2
  // Ask the child to close the read end of the error pipe.
  checkUnixError(fcntl(errFds[0], F_SETFD, FD_CLOEXEC), "set FD_CLOEXEC");
  // Set the close-on-exec flag on the write side of the pipe.
  // This way the pipe will be closed automatically in the child if execve()
  // succeeds.  If the exec fails the child can write error information to the
  // pipe.
  checkUnixError(fcntl(errFds[1], F_SETFD, FD_CLOEXEC), "set FD_CLOEXEC");
#endif

  // Perform the actual work of setting up pipes then forking and
  // executing the child.
  spawnInternal(std::move(argv), executable, options, env, errFds[1]);

  // After spawnInternal() returns the child is alive.  We have to be very
  // careful about throwing after this point.  We are inside the constructor,
  // so if we throw the Subprocess object will have never existed, and the
  // destructor will never be called.
  //
  // We should only throw if we got an error via the errFd, and we know the
  // child has exited and can be immediately waited for.  In all other cases,
  // we have no way of cleaning up the child.

  // Close writable side of the errFd pipe in the parent process
  CHECK_ERR(::close(errFds[1]));
  errFds[1] = -1;

  // Read from the errFd pipe, to tell if the child ran into any errors before
  // calling exec()
  readChildErrorPipe(errFds[0], executable);

  // If we spawned a detached child, wait on the intermediate child process.
  // It always exits immediately.
  if (options.detach_) {
    wait();
  }

  // We have fully succeeded now, so release the guard on pipes_
  pipesGuard.dismiss();
}

FOLLY_PUSH_WARNING void folly::Subprocess::spawnInternal ( std::unique_ptr< const char *[]>  argv, const char *  executable, Options &  options, const std::vector< std::string > *  env, int  errFd  )

private

Definition at line 327 of file Subprocess.cpp.

References folly::checkPosixError(), folly::checkUnixError(), folly::Subprocess::Pipe::childFd, folly::netops::close(), folly::Subprocess::Pipe::direction, gmock_leak_test::environ, folly::errnoStr(), folly::SubprocessSpawnError::errnoValue(), FOLLY_POP_WARNING, kChildFailure, kExecFailure, folly::ProcessReturnCode::makeRunning(), pipe(), folly::Subprocess::Pipe::pipe, runChild(), and SCOPE_EXIT.

               {
  // Parent work, pre-fork: create pipes
  std::vector<int> childFds;
  // Close all of the childFds as we leave this scope
  SCOPE_EXIT {
    // These are only pipes, closing them shouldn't fail
    for (int cfd : childFds) {
      CHECK_ERR(::close(cfd));
    }
  };

  int r;
  for (auto& p : options.fdActions_) {
    if (p.second == PIPE_IN || p.second == PIPE_OUT) {
      int fds[2];
      // We're setting both ends of the pipe as close-on-exec. The child
      // doesn't need to reset the flag on its end, as we always dup2() the fd,
      // and dup2() fds don't share the close-on-exec flag.
#if FOLLY_HAVE_PIPE2
      // If possible, set close-on-exec atomically. Otherwise, a concurrent
      // Subprocess invocation can fork() between "pipe" and "fnctl",
      // causing FDs to leak.
      r = ::pipe2(fds, O_CLOEXEC);
      checkUnixError(r, "pipe2");
#else
      r = ::pipe(fds);
      checkUnixError(r, "pipe");
      r = fcntl(fds[0], F_SETFD, FD_CLOEXEC);
      checkUnixError(r, "set FD_CLOEXEC");
      r = fcntl(fds[1], F_SETFD, FD_CLOEXEC);
      checkUnixError(r, "set FD_CLOEXEC");
#endif
      pipes_.emplace_back();
      Pipe& pipe = pipes_.back();
      pipe.direction = p.second;
      int cfd;
      if (p.second == PIPE_IN) {
        // Child gets reading end
        pipe.pipe = folly::File(fds[1], /*ownsFd=*/true);
        cfd = fds[0];
      } else {
        pipe.pipe = folly::File(fds[0], /*ownsFd=*/true);
        cfd = fds[1];
      }
      p.second = cfd; // ensure it gets dup2()ed
      pipe.childFd = p.first;
      childFds.push_back(cfd);
    }
  }

  // This should already be sorted, as options.fdActions_ is
  DCHECK(std::is_sorted(pipes_.begin(), pipes_.end()));

  // Note that the const casts below are legit, per
  // http://pubs.opengroup.org/onlinepubs/009695399/functions/exec.html

  char** argVec = const_cast<char**>(argv.get());

  // Set up environment
  std::unique_ptr<const char*[]> envHolder;
  char** envVec;
  if (env) {
    envHolder = cloneStrings(*env);
    envVec = const_cast<char**>(envHolder.get());
  } else {
    envVec = environ;
  }

  // Block all signals around vfork; see http://ewontfix.com/7/.
  //
  // As the child may run in the same address space as the parent until
  // the actual execve() system call, any (custom) signal handlers that
  // the parent has might alter parent's memory if invoked in the child,
  // with undefined results.  So we block all signals in the parent before
  // vfork(), which will cause them to be blocked in the child as well (we
  // rely on the fact that Linux, just like all sane implementations, only
  // clones the calling thread).  Then, in the child, we reset all signals
  // to their default dispositions (while still blocked), and unblock them
  // (so the exec()ed process inherits the parent's signal mask)
  //
  // The parent also unblocks all signals as soon as vfork() returns.
  sigset_t allBlocked;
  r = sigfillset(&allBlocked);
  checkUnixError(r, "sigfillset");
  sigset_t oldSignals;

  r = pthread_sigmask(SIG_SETMASK, &allBlocked, &oldSignals);
  checkPosixError(r, "pthread_sigmask");
  SCOPE_EXIT {
    // Restore signal mask
    r = pthread_sigmask(SIG_SETMASK, &oldSignals, nullptr);
    CHECK_EQ(r, 0) << "pthread_sigmask: " << errnoStr(r); // shouldn't fail
  };

  // Call c_str() here, as it's not necessarily safe after fork.
  const char* childDir =
      options.childDir_.empty() ? nullptr : options.childDir_.c_str();

  pid_t pid;
#ifdef __linux__
  if (options.cloneFlags_) {
    pid = syscall(SYS_clone, *options.cloneFlags_, 0, nullptr, nullptr);
  } else {
#endif
    if (options.detach_) {
      // If we are detaching we must use fork() instead of vfork() for the first
      // fork, since we aren't going to simply call exec() in the child.
      pid = fork();
    } else {
      pid = vfork();
    }
#ifdef __linux__
  }
#endif
  checkUnixError(pid, errno, "failed to fork");
  if (pid == 0) {
    // Fork a second time if detach_ was requested.
    // This must be done before signals are restored in prepareChild()
    if (options.detach_) {
#ifdef __linux__
      if (options.cloneFlags_) {
        pid = syscall(SYS_clone, *options.cloneFlags_, 0, nullptr, nullptr);
      } else {
#endif
        pid = vfork();
#ifdef __linux__
      }
#endif
      if (pid == -1) {
        // Inform our parent process of the error so it can throw in the parent.
        childError(errFd, kChildFailure, errno);
      } else if (pid != 0) {
        // We are the intermediate process.  Exit immediately.
        // Our child will still inform the original parent of success/failure
        // through errFd.  The pid of the grandchild process never gets
        // propagated back up to the original parent.  In the future we could
        // potentially send it back using errFd if we needed to.
        _exit(0);
      }
    }

    int errnoValue = prepareChild(options, &oldSignals, childDir);
    if (errnoValue != 0) {
      childError(errFd, kChildFailure, errnoValue);
    }

    errnoValue = runChild(executable, argVec, envVec, options);
    // If we get here, exec() failed.
    childError(errFd, kExecFailure, errnoValue);
  }

  // Child is alive.  We have to be very careful about throwing after this
  // point.  We are inside the constructor, so if we throw the Subprocess
  // object will have never existed, and the destructor will never be called.
  //
  // We should only throw if we got an error via the errFd, and we know the
  // child has exited and can be immediately waited for.  In all other cases,
  // we have no way of cleaning up the child.
  pid_ = pid;
  returnCode_ = ProcessReturnCode::makeRunning();
}

int folly::Subprocess::stderrFd ( ) const

inline

Definition at line 878 of file Subprocess.h.

                       {
    return parentFd(2);
  }

int folly::Subprocess::stdinFd ( ) const

inline

Definition at line 872 of file Subprocess.h.

                      {
    return parentFd(0);
  }

int folly::Subprocess::stdoutFd ( ) const

inline

Definition at line 875 of file Subprocess.h.

Referenced by TEST().

                       {
    return parentFd(1);
  }

std::vector< Subprocess::ChildPipe > folly::Subprocess::takeOwnershipOfPipes

(

)

Definition at line 907 of file Subprocess.cpp.

References folly::gen::move, and folly::swap().

Referenced by TEST().

                                                                {
  std::vector<Subprocess::ChildPipe> pipes;
  for (auto& p : pipes_) {
    pipes.emplace_back(p.childFd, std::move(p.pipe));
  }
  // release memory
  std::vector<Pipe>().swap(pipes_);
  return pipes;
}

void folly::Subprocess::terminate ( )

inline

Definition at line 608 of file Subprocess.h.

                   {
    sendSignal(SIGTERM);
  }

ProcessReturnCode folly::Subprocess::wait

(

)

Wait for the process to terminate and return its status. Like poll(), the only exception this can throw is std::logic_error if you call this on a Subprocess whose status is RUNNING. Aborts on egregious violations of contract, like an out-of-band waitpid(p.pid(), 0, 0).

Definition at line 644 of file Subprocess.cpp.

References folly::ProcessReturnCode::enforce(), folly::ProcessReturnCode::make(), and folly::ProcessReturnCode::RUNNING.

Referenced by TEST().

                                   {
  returnCode_.enforce(ProcessReturnCode::RUNNING);
  DCHECK_GT(pid_, 0);
  int status;
  pid_t found;
  do {
    found = ::waitpid(pid_, &status, 0);
  } while (found == -1 && errno == EINTR);
  // The only two remaining errors are ECHILD (other code reaped the
  // child?), or EINVAL (cosmic rays?), and both merit an abort:
  PCHECK(found != -1) << "waitpid(" << pid_ << ", &status, WNOHANG)";
  // Though the child process had quit, this call does not close the pipes
  // since its descendants may still be using them.
  DCHECK_EQ(found, pid_);
  returnCode_ = ProcessReturnCode::make(status);
  pid_ = -1;
  return returnCode_;
}

void folly::Subprocess::waitChecked

(

)

Wait for the process to terminate, throw if unsuccessful.

Definition at line 663 of file Subprocess.cpp.

References folly::detail::distributed_mutex::wait().

Referenced by TEST().

                             {
  wait();
  checkStatus(returnCode_);
}

Member Data Documentation

const int folly::Subprocess::CLOSE = -1

static

Definition at line 271 of file Subprocess.h.

pid_t folly::Subprocess::pid_ {-1}

private

Definition at line 941 of file Subprocess.h.

const int folly::Subprocess::PIPE = -2

static

Definition at line 272 of file Subprocess.h.

Referenced by folly::Subprocess::Options::fd().

const int folly::Subprocess::PIPE_IN = -3

static

Definition at line 273 of file Subprocess.h.

Referenced by folly::Subprocess::Options::fd().

const int folly::Subprocess::PIPE_OUT = -4

static

Definition at line 274 of file Subprocess.h.

Referenced by folly::Subprocess::Options::fd().

std::vector<Pipe> folly::Subprocess::pipes_

private

Definition at line 970 of file Subprocess.h.

ProcessReturnCode folly::Subprocess::returnCode_

private

Definition at line 942 of file Subprocess.h.

---

The documentation for this class was generated from the following files:

• proxygen/folly/folly/Subprocess.h
• proxygen/folly/folly/Subprocess.cpp