wangle::MultiFilePoller Class Reference

#include <MultiFilePoller.h>

Classes
struct	CallbackDetail
class	CallbackId

Public Types
using	CallbackArg = std::unordered_map< std::string, std::string >
using	Callback = folly::Function< void(const CallbackArg &newData) noexcept >

Public Member Functions
	MultiFilePoller (std::chrono::milliseconds pollInterval)
	~MultiFilePoller ()=default
CallbackId	registerFile (std::string path, Callback cb)
CallbackId	registerFiles (const std::vector< std::string > &paths, Callback cb)
void	cancelCallback (const CallbackId &cbId)

Private Types
using	StringReferences = std::vector< std::reference_wrapper< const std::string >>

Private Member Functions
void	onFileUpdated (const std::string &triggeredPath)
size_t	getNextCallbackId ()

Private Attributes
folly::SharedMutex	rwlock_
size_t	lastCallbackId_ = 0
std::unordered_map< std::string, std::vector< size_t > >	pathsToCallbackIds_
std::unordered_map< size_t, CallbackDetail >	idsToCallbacks_
wangle::FilePoller	poller_

Detailed Description

An extension to wangle::FilePoller with the ability to register one or more callback on a file, and to track one or more file in a callback, and to deliver cached file data to callbacks.

Definition at line 32 of file MultiFilePoller.h.

Member Typedef Documentation

using wangle::MultiFilePoller::Callback = folly::Function<void(const CallbackArg& newData) noexcept>

Definition at line 58 of file MultiFilePoller.h.

using wangle::MultiFilePoller::CallbackArg = std::unordered_map<std::string, std::string>

A callback: (1) takes as argument a map from a file path to its latest content. Unreadable paths will not show up in the map. (2) is triggered when any file it registers is changed, in the context of FilePoller thread. (3) once registered, cannot have its file list or callback pointer changed. To make changes, cancel the existing one and register a new one. Caveat: (1) If there are N files registered, in worst case the first N-1 callbacks would not get latest content of all N files. It's up to the callback to determine what to do for those cases. (2) Once a file, say, F, changes, not only F, but all files needed to trigger all callbacks that use F will be read. For example, Callback A needs files {a, b}; Callback B needs files {b, c}; Callback C needs files {d, e}; File {a} changes -> Reads {a, b} -> Calls A(a, b). File {b} changes -> Reads {a, b, c} -> Calls A(a, b), B(b, c). File {c} changes -> Reads {b, c} -> Calls B(b, c). File {d} or {e} changes -> Reads {d, e} -> Calls C(d, e).

Definition at line 57 of file MultiFilePoller.h.

using wangle::MultiFilePoller::StringReferences = std::vector<std::reference_wrapper<const std::string>>

private

Definition at line 115 of file MultiFilePoller.h.

Constructor & Destructor Documentation

wangle::MultiFilePoller::MultiFilePoller ( std::chrono::milliseconds  pollInterval )

explicit

Parameters

pollInterval

Interval between polls. Setting a value less than 1_s may cause undesirable behavior because the minimum granularity for wangle::FilePoller to detect mtime difference is 1_s.

Definition at line 27 of file MultiFilePoller.cpp.

    : poller_(pollInterval) {}

wangle::MultiFilePoller::~MultiFilePoller ( )

default

Member Function Documentation

void wangle::MultiFilePoller::cancelCallback

(

const CallbackId &

cbId

)

Cancel the specified Callback. May throw std::out_of_range if not found.

Parameters

cbId	ID of the callback returned when registering it.

Definition at line 72 of file MultiFilePoller.cpp.

References wangle::MultiFilePoller::CallbackId::id_, idsToCallbacks_, pathsToCallbackIds_, poller_, wangle::FilePoller::removeFileToTrack(), and rwlock_.

                                                           {
  std::vector<std::string> pathsToErase;
  SharedMutex::WriteHolder wh(rwlock_);

  auto pos = idsToCallbacks_.find(cbId.id_);
  if (pos == idsToCallbacks_.end()) {
    throw std::out_of_range(
        to<std::string>("Callback ", cbId.id_, " not found"));
  }

  // Remove the callback ID from its registered paths.
  for (const auto& path : pos->second.files_) {
    auto& callbackIds = pathsToCallbackIds_[path];
    callbackIds.erase(
        std::remove(callbackIds.begin(), callbackIds.end(), cbId.id_));
    // If the path has no more callbacks, erase it from map.
    if (callbackIds.empty()) {
      poller_.removeFileToTrack(path);
      pathsToErase.emplace_back(path);
    }
  }
  // Remove the callback.
  idsToCallbacks_.erase(cbId.id_);
  // Remove callback-less paths from pathsToCallbackIds_, if any, at last.
  for (const auto& path : pathsToErase) {
    pathsToCallbackIds_.erase(path);
  }
}

size_t wangle::MultiFilePoller::getNextCallbackId ( )

private

Find an unused size_t value as callback Id. Caller must acquire wlock.

Definition at line 30 of file MultiFilePoller.cpp.

References idsToCallbacks_, and lastCallbackId_.

Referenced by registerFiles().

                                          {
  size_t ret = lastCallbackId_;
  // Order of callback cancellation is arbitrary, so the next available
  // callbackId might not be lastCallbackId_+1.
  while (++ret != lastCallbackId_) {
    if (idsToCallbacks_.find(ret) == idsToCallbacks_.end()) {
      lastCallbackId_ = ret; // Assume write lock is acquired.
      return ret;
    }
  }
  throw std::runtime_error("Run out of callback ID.");
}

void wangle::MultiFilePoller::onFileUpdated ( const std::string &  triggeredPath )

private

The callback dispatcher to be registered to wangle::FilePoller.

Definition at line 101 of file MultiFilePoller.cpp.

References data, idsToCallbacks_, folly::gen::move, pathsToCallbackIds_, folly::readFile(), rwlock_, and string.

Referenced by registerFiles().

                                                                  {
  VLOG(4) << "onFileUpdated(" << triggeredPath << ").";

  // A temporary read cache. Not worth it making it permanent because
  // files do not change frequently.
  std::unordered_map<std::string, std::string> filePathsToFileContents;
  SharedMutex::ReadHolder rh(rwlock_);

  const auto& callbacks = pathsToCallbackIds_.find(triggeredPath);
  if (callbacks == pathsToCallbackIds_.end()) {
    return;
  }

  for (const auto& cbId : callbacks->second) {
    const auto& cbEnt = idsToCallbacks_.find(cbId);
    // Lazily read all files needed by the callback.
    for (const auto& path : cbEnt->second.files_) {
      if (filePathsToFileContents.find(path) == filePathsToFileContents.end()) {
        std::string data;
        if (readFile(path.get().c_str(), data)) {
          filePathsToFileContents.emplace(path, std::move(data));
        } else {
          VLOG(4) << "Failed to read file " << path.get();
        }
      }
    }
    cbEnt->second.cb_(filePathsToFileContents);
  }
}

MultiFilePoller::CallbackId wangle::MultiFilePoller::registerFile	(	std::string	path,
		Callback	cb
	)

Add a callback to trigger when the specified file changes.

Parameters

path	The path to monitor.
cb	The callback to trigger.

Returns

An ID of the callback, used for cancellation.

Definition at line 43 of file MultiFilePoller.cpp.

References folly::gen::move, and registerFiles().

                 {
  return registerFiles({std::move(path)}, std::move(cb));
}

MultiFilePoller::CallbackId wangle::MultiFilePoller::registerFiles	(	const std::vector< std::string > &	paths,
		Callback	cb
	)

Add a callback to trigger when any of the files in paths changes.

Parameters

paths	The list of paths to monitor. Must be non-empty.
cb	The callback to trigger. @ return An ID of the callback, used for cancellation.

Definition at line 49 of file MultiFilePoller.cpp.

References wangle::FilePoller::addFileToTrack(), getNextCallbackId(), idsToCallbacks_, join, folly::gen::move, onFileUpdated(), pathsToCallbackIds_, poller_, and rwlock_.

Referenced by registerFile().

                 {
  VLOG(4) << "registerFiles({" << join(", ", paths) << "}, cb=" << &cb << ")";
  if (paths.empty()) {
    throw std::invalid_argument("Argument paths must be non-empty.");
  }
  StringReferences cbPaths;
  SharedMutex::WriteHolder wh(rwlock_);
  auto cbId = getNextCallbackId();
  // Create the bi-directional relation between path and callback.
  for (const auto& path : paths) {
    pathsToCallbackIds_[path].push_back(cbId);
    // Use reference to key of pathsToCallbackIds_ map to avoid duplicates.
    const auto& key = pathsToCallbackIds_.find(path)->first;
    cbPaths.push_back(key);
    poller_.addFileToTrack(key, [=] { onFileUpdated(key); });
  }
  idsToCallbacks_.emplace(cbId,
                          CallbackDetail(std::move(cbPaths), std::move(cb)));
  return MultiFilePoller::CallbackId(cbId);
}

Member Data Documentation

std::unordered_map<size_t, CallbackDetail> wangle::MultiFilePoller::idsToCallbacks_

private

Definition at line 129 of file MultiFilePoller.h.

Referenced by cancelCallback(), getNextCallbackId(), onFileUpdated(), and registerFiles().

size_t wangle::MultiFilePoller::lastCallbackId_ = 0

private

Definition at line 127 of file MultiFilePoller.h.

Referenced by getNextCallbackId().

std::unordered_map<std::string, std::vector<size_t> > wangle::MultiFilePoller::pathsToCallbackIds_

private

Definition at line 128 of file MultiFilePoller.h.

Referenced by cancelCallback(), onFileUpdated(), and registerFiles().

wangle::FilePoller wangle::MultiFilePoller::poller_

private

Definition at line 132 of file MultiFilePoller.h.

Referenced by cancelCallback(), and registerFiles().

folly::SharedMutex wangle::MultiFilePoller::rwlock_

private

Definition at line 126 of file MultiFilePoller.h.

Referenced by cancelCallback(), onFileUpdated(), and registerFiles().

---

The documentation for this class was generated from the following files:

• proxygen/wangle/wangle/util/MultiFilePoller.h
• proxygen/wangle/wangle/util/MultiFilePoller.cpp